Job Title: Lead Data Engineer

Company Name: Acklen Avenue

Job Url: https://www.simplyhired.com/job/aDlBRV2uvS3rEQB7d9rLetLc-csybsAZXn8wEyWkOe32EWFirg8UUw

Job Description: Lead Data Engineer (Flink)
Acklen Avenue
Remote

Job Details
Full-time | Contract
Qualifications
Data model design
Microsoft Certified: Azure Fundamentals
Data modeling
Data Integration (Data management)
Internet of Things
5 years
Scalable systems
Java
SQL
Data quality management
Database design
Solution architecture design
Continuous improvement
Azure DevOps proficiency
Azure AD
Software documentation
GitHub
APIs
Scalability
Agile software development
Kafka
Real-time data processing implementation
Senior level
Microsoft Certified: Azure Data Engineer Associate
Cloud services
System performance monitoring
Database software proficiency
Full Job Description
LEAD DATA ENGINEER (FLINK) - LATAM ONLY

You’re a seasoned Lead Data Engineer with 5+ years of experience designing and delivering real-time streaming solutions in the Azure ecosystem, including deep expertise in Java and Apache Flink (DataStream API and SQL). You have hands-on experience with stateful processing, checkpointing, watermark strategies, and complex IoT data patterns such as out-of-order and missing events, and you can build scalable, high-performance pipelines using Flink 2.x with Azure Event Hubs or Kafka. Proficient in Java and SQL, you understand Azure data services, data modeling, governance, and security best practices, and you thrive in Agile environments, leveraging CI/CD and DevOps tools to deliver reliable, cost-efficient data solutions while promoting strong engineering standards and collaboration.

POSITION ACCOUNTABILITIES

● Architect, design, and implement scalable and efficient data solutions on Flink.

● Design and implement the Flink 2.x streaming pipeline for IoT.

● Develop custom KeyedProcessFunctions to handle "Heartbeat Injection" (detecting sensor silence) and complex windowing.

● Implement the Gatekeeper Pattern: A Flink job that applies Data Quality tags (Validity,

Completeness) to the stream without dropping data.

● Optimize State Backends for high-cardinality sensor data (100k+ concurrent sensors).

● Work with the "Shared Core" team to integrate libraries into the Flink runtime.

● Manage and optimize Azure resources (Flink clusters) for performance, reliability, and

cost-efficiency.

● Transform, clean, and prepare data using SQL and Java.

● Monitor and fine-tune workloads and pipelines for optimal performance and reliability.

● Maintain clear documentation of solutions, configurations, and workflows.

● Actively participate in Agile team activities and continuous improvement initiatives.

● Promote and enforce data engineering best practices, including data governance, security, and data quality.

POSITION SKILLS & REQUIREMENTS

Must have

● 5+ years of hands-on data engineering experience with deep expertise in the Azure

ecosystem.

● 5+ years in Java (backend) with 3+ years deep specialization in Apache Flink (DataStream API

& SQL).

● Experience with Flink State Management, Checkpointing, and Watermark strategies.

● Experience upgrading to or using Flink 2.0/2.x.

● Experience with Azure Event Hubs or Kafka.

● Understanding of "Sensor/IoT" data patterns (out-of-order and missing events).

● Proficient in Java, SQL, and writing optimized data integration and processing code.

● Strong understanding of SDLC and Agile methodologies with hands-on experience in Azure DevOps, GitHub, CI/CD, and artifact management.

● Deep expertise in Azure data services.

● Skilled in data modeling, database design, and data warehousing solutions on Azure.

● Knowledge of data quality, governance, and security best practices within Azure (AD, NSG, encryption, compliance).

Nice to have

● Certifications preferred: Azure Fundamentals, Azure Data Engineer Associate, and Azure

Solutions Architect Expert (nice to have).

Job Types: Full-time, Contract

Work Location: Remote