Job Title: Data Processing Principal Engineer

Company Name: DataPelago

Job Details: RemoteFull,Time

Job Url: https://hiring.cafe/viewjob/7b3xpadl0wsslpey

Job Description: Posted 10mo agoData Processing Principal Engineer  @ DataPelagoView All JobsWebsiteNorth AmericaRemoteFull TimeResponsibilities:leading architecture, designing engine, collaborating teamsRequirements Summary:15+ years in EE/CS with 10+ years in enterprise data engines; strong C/C++/Rust; Linux; leadership of large teams.Technical Tools Mentioned:Apache Spark, Velox, DataFusion, C, C++, Rust, Linux
Lead architecture, design, and implementation of next-generation data processing engine built to  exploit accelerated computing.  
The developer will have the following specific responsibilities in achieving these objectives.  Responsibilities  
• Work with cross-functional teams to advance the architecture at the core of Datapelago’s  parallel and distributed execution engine based on accelerated computing 
• Lead the execution engine team in the design, implementation, and rollout of enterprise grade, highly reliable acceleration engine for data processing 
• Design, implement, test, and maintain major components of the execution engine • Analyze and identify areas for differentiation and improvements in the execution engine • Collaborate with other teams and team members to drive code reviews, design reviews,  
performance and reliability reviews, and development process and to drive continuous  improvement in all of these 
Qualifications 
• B.S. EE/CS or equivalent with 15+ years experience or MS with 10+ years experience • 10+ years of experience developing core components of an enterprise-grade database or  analytics execution engine serving large-scale data processing workloads. Experience  developing for platforms such as Apache Spark, Gluten, Velox, DataFusion preferred.  • Experienced developing high-performance parallel implementations of data processing  operators and functions, such as joins, aggregations, sorts  
• Experience leading 10+ teams in designing, developing, and releasing high-performance  data processing engines for large production deployments 
• Strong programming ability in C, C++, and Rust 
• Strong development experience on Linux platforms