Job Title: Senior Data Engineer

Company Name: Clara Analytics

Job Details: $160k-$190k/yrRemoteFull,Time

Job Url: https://hiring.cafe/viewjob/cz4mxbtpgryhnaxb

Job Description: Posted 1d agoSenior Data Engineer@ Clara AnalyticsView All JobsUnited States$160k-$190k/yrRemoteFull TimeResponsibilities:architect pipelines, ingest data, collaborate teamsRequirements Summary:6+ years building production data pipelines with Spark/PySpark; 4+ years AWS data engineering; 4+ years Python/Scala/Go; 5+ years AWS data solutions; 3+ years container orchestration; 2+ years data quality frameworks; strong SQL; GitOps/IaC practices.Technical Tools Mentioned:Spark, PySpark, Spark SQL, Airflow, AWS Step Functions, Prefect, Glue, EMR, Lambda, S3, Redshift, Docker, Kubernetes, EKS, ECR, Databrew, Glue Data Quality, Pandas, NumPy
About the RoleJoin CLARA Analytics as a Senior Data Engineer and help transform the insurance industry by solving complex data challenges at scale. You'll build next-generation data infrastructure using cutting-edge AWS technologies that power our revolutionary AI models - from ingesting claim documents to architecting pipelines that process medical claims. If you're passionate about leveraging modern tools like Spark, PySpark Glue, and the full AWS ecosystem to tackle problems that directly impact claims adjusters, this is your opportunity to be part of something truly transformative in an industry ripe for disruption.What You'll Do...Architect and implement modern, scalable ETL/ELT pipelines using modern AWS-native services to process insurance claims data.Build resilient, high-throughput data pipelines with an emphasis on quality and reliability to drive consistent, accurate data across the enterprise.Ingest and transform diverse data sources, structured, semi-structured and unstructured, into enterprise-level ETL/ELT solutions.Design and implement custom algorithms to solve complex data challenges and unlock new insights. Collaborate cross-functionally with Data Scientists, Analysts, and Product teams to deliver business-aligned solutions.Ensure pipeline reliability, performance, and SLA adherence.Streamline operations through automation, CI/CD, infrastructure-as-code, and configuration management tools.What We’re Looking For...Required6+ years building production data pipelines with Spark, PySpark, and Spark SQL, plus orchestration experience with Airflow, AWS Step Functions, or Prefect4+ years deep AWS data engineering expertise including tools such as Glue, EMR, Spark, and Lake Formation4+ years mastering Python (or Scala or Go) for data engineering with experience in modern frameworks like pandas, numpy, and data validation libraries5+ years architecting data solutions using AWS services, from S3 data lakes to Redshift warehouses, with Glue ETL and Lambda for serverless processing3+ years with container orchestration (Docker, Kubernetes, EKS) for scalable data workloads and microservices2+ years implementing data quality frameworks using AWS Glue Data Quality or Databrew. Expert-level SQL skills including advanced analytics, window functions, and query optimization for large-scale data processingStrong engineering practices including GitOps workflows, infrastructure-as-code (Terraform/CDK), automated testing, and DataOps methodologiesProduction containerization experience with Docker, Kubernetes, Helm, and AWS container services (EKS, ECS, ECR)Thrives in fast-paced environments with excellent collaboration skills and adaptability to evolving requirementsCreative problem-solver with strong debugging skills and ability to architect innovative solutions for complex data challengesPreferredDirect experience with insurance claims data systems, including claims management platforms (Guidewire, Duck Creek, or Majesco), or healthcare EDI standardsExperience working with medical coding systems (ICD-10, CPT, NDC) and extracting insights from healthcare claims dataAWS professional certifications (Solutions Architect, Data Analytics Specialty, or DevOps Professional) demonstrating cloud architecture expertiseBackground in insurance or healthcare analytics where you've built data solutions that improved underwriting accuracy, claims processing efficiency, or fraud detectionWhat We Offer...The opportunity to make a real impact on a growing company.Collaborative and supportive work environment.Competitive compensation package.Salary of $160 - 190k + 10% BonusBenefits: employer-provided health insurance and ancillary benefits (life, disability, etc.), flexible PTO, fully remote, 401k with match, etc. Be a part of a team that is passionate about what we do!