Job Url: https://www.remoterocketship.com/company/informa/jobs/staff-engineer-%E2%80%93-data-platform-and-lakehouse-united-states Job Description: Informa Website LinkedIn All Job Openings Informa is a global leader in business intelligence, academic publishing, and events. It is home to hundreds of specialist brands and businesses, focused on connecting people with knowledge through events and digital products. Informa operates in over 30 countries and has a workforce of 12,000 colleagues globally. The company runs divisions such as Informa Markets, which facilitates trade and growth through events, and Taylor & Francis, a leading publisher of academic research. Informa prioritizes sustainability and diversity, aiming to positively impact markets and communities. As a member of the FTSE 100, Informa is known for its strong financial performance and scale within the academic and B2B markets. Academic Publishing • Marketing Services • Data & Intelligence • Digital events • Exhibitions 10,000+ employees Founded 1998 🤝 B2B 📱 Media 📚 Education Staff Engineer - Data Platform and Lakehouse 5 hours ago 🗽 New York – Remote 💵 $180k - $220k / year ⏰ Full Time 🔴 Lead 🧑‍💻 Full-stack Engineer 🦅 H1B Visa Sponsor Airflow Apache AWS Azure Cloud ETL Google Cloud Platform Python SDLC Spark SQL Unity Apply Now Receive Emails with Similar Jobs Report problem 📋 Description • Design and implement scalable, secure, and maintainable data platforms on Databricks and AWS cloud infrastructure • Provide architectural leadership across engineering domains, ensuring consistency, scalability, and resilience • Architect distributed data processing systems using Apache Spark and optimize for performance and scalability • Lead development of reusable data pipelines and workflows using Databricks Workflows • Translate business objectives into platform capabilities in collaboration with Product Managers and cross-functional teams • Support AI/ML initiatives through robust data engineering, including feature engineering, model deployment • Champion best practices in ETL/ELT, data quality, monitoring, observability, and Agile development • Drive adoption of data governance standards: access control, metadata management, lineage, and compliance • Establish and maintain CI/CD pipelines and DevOps automation for data infrastructure • Evaluate and integrate emerging technologies to enhance development, testing, deployment, and monitoring 🎯 Requirements • 15+ years of experience in software development, covering the full SDLC: ideation, design, development, testing, deployment, and support • Strategic mindset, capable of aligning technical decisions with business goals and driving architectural vision • Excellent collaboration and communication skills across functions, including data scientists, MLOps engineers, and product managers • Advanced proficiency in Python and SQL, with a strong foundation in software engineering principles • Extensive experience with distributed computing frameworks, particularly Apache Spark, including performance tuning and scalability • Required hands-on experience with Databricks, including Unity Catalog, Feature Store, and Delta Live Tables • Proficiency in data pipeline orchestration tools using Databricks Workflows, Airflow, or AWS Glue; Databricks required • Strong command of data engineering best practices and tools, including ETL/ELT design, data quality validation, monitoring, and observability • Experience with Monte Carlo preferred • Proven ability to build scalable, cloud-native data architectures on Databricks, AWS, Azure, or GCP; Databricks required • Experience enabling AI/ML workloads including feature engineering, model deployment, and real-time processing • Strong understanding of data governance and security, including access control, data lineage, compliance, and metadata management 🏖️ Benefits • Competitive benefits including Financial, Health and Lifestyle benefits to choose from • Flexible working options including home working, flexible hours and part time options depending on role requirements – please ask! • Competitive annual leave, floating holidays, volunteering days and a day off for your birthday • Learning and development tools to assist with your career development • Work with industry leading Subject Matter Experts and specialist products • Regular social events and networking opportunities • Collaborative, supportive culture, including an active DE&I program • Employee Assistance Program which provides expert third-party advice on wellbeing, relationships, legal and financial matters, as well as access to counselling services