Job Title: Senior Data Engineer

Company Name: Middesk

Job Url: https://jobs.ashbyhq.com/middesk/dfaf630c-aa58-406b-be18-020f21ecc040?utm_source=jobright&jr_id=69a772b376bbe83d3afad6a0

Job Description: The Role:

We're looking for a senior engineer to own and drive the technical direction of our data platform. You'll design data infrastructure systems that operate reliably at scale, define the architecture for how we acquire, transform, and serve data to product teams, and partner closely with engineering leadership to align platform strategy with company priorities.

You'll be expected to independently scope complex projects from ambiguous problem statements, break them into incremental deliverables, and drive them to completion while keeping stakeholders informed and adapting when priorities shift. You'll also play a key role in elevating the engineering practices of the team through mentorship, code review, and setting technical standards.

What You'll Do:

Own platform architecture and technical direction for how we ingest, transform, and serve data across highly variable input formats - business entity data sourced from thousands of government agencies, registries, and third-party providers, each with its own schema, cadence, and reliability profile

Design and build systems for scale - both the infrastructure we need today and the infrastructure we'll need at 2–5x our current volume

Scope and drive complex projects end-to-end, breaking ambiguous problems into well-defined milestones with clear deliverables and timelines

Design AI-powered tooling to improve how we acquire and maintain data using LLMs, AI agents, and agent orchestration

Partner with product engineering, data science, and business teams to understand data needs and translate them into platform capabilities

Establish and maintain data governance and quality standards across the platform, ensuring the integrity and reliability of the data our customers depend on for compliance and risk decisions

What We’re Looking For:

7+ years of professional software engineering experience, with meaningful time spent on data infrastructure, data engineering, or backend platform work (targeting Senior to Staff Level Engineers)

Experience designing and operating systems at meaningful scale, ideally within a larger or rapidly scaling engineering organization

Track record of independently owning and delivering complex, multi-milestone projects - from scoping through launch

Strong data modeling instincts and deep familiarity with SQL, pipeline orchestration (Airflow, Dagster, etc.), and data transformation patterns

Experience with distributed data processing frameworks (Spark, Flink, Beam, or similar) and an understanding of when and how to apply parallelization to scale pipelines beyond single-node limits

Proficiency in one or more of: Python, Ruby, JavaScript/TypeScript, Java

Nice to Haves:

Experience working with or building AI/LLM-powered tooling or data products (strongly preferred)

Experience with scraper technologies, including agentic AI

Experience building and designing collections stored on Elasticsearch

Experience operating event-driven data pipelines using serverless compute (e.g., AWS Lambda, Google Cloud Functions, Azure Functions) and managed cloud services

Experience with Terraform, Datadog, or Kubernetes

Apply for this Job