Job Title: Software Engineer - Data Ingestion Team

Company Name: LightBox

Job Details: RemoteFull,Time

Job Url: https://hiring.cafe/viewjob/7ox5xn9n3f13i38l

Job Description: Posted 4y agoSoftware Engineer - Data Ingestion Team@ LightBoxView All JobsWebsiteUnited StatesRemoteFull TimeResponsibilities:Develop ingestion, Review design, Maintain pipelinesRequirements Summary:BS in CS; 2-5 years software engineering; Python; OOP; SQL; data ingestion; cloud experience.Technical Tools Mentioned:Python, SQL, MySQL, MS-SQL, Prefect, Luigi, Airflow, Hadoop, Spark, pandas, Dask, Docker, Kubernetes, Azure, AWS, Google Cloud, Flask, React, Linux
Position Overview:We are currently looking for a keen Software Engineer to join our data ingestion team. Working on a greenfield project as part of a world class data platform; this team does exciting work in the area of highly scalable data ingestion pipelines for location-based analytics. You will be building the automation and tooling to support retrieval and transformation of tens of thousands of externally sourced datasets. You will work in a team with highly qualified and accomplished data and software engineers to build, enhance, and maintain our data platform that supports our best in class products.What you will do and achieve:Reporting to the Software Development Manager, the duties and responsibilities of the Software Engineer include, but are not limited to:Work with an agile team to develop, test, and maintain data ingestion applications and workflowsParticipate in design reviews and pull requestsAdhere to high-quality development principles while delivering solutions on-time and on-budgetAnalyze and resolve technical and application problemsContribute to our evolving Continuous Integration (CI/CD) pipelineAnalyze use cases and propose solutions to meet business objectivesWho you are:EducationBachelor's degree or better in Computer ScienceMust have an excellent academic record with a good grounding in Software Engineering theory including at least one modern programming languageFamiliar with Standards, concepts, practices, and procedures within the field of Computer ScienceExperience2 to 5 years’ experience as a software engineerKey Knowledge & SkillsAt least one modern programming language, preferably PythonKnowledge and experience with OOP design patternsAny RDBMS, such as MySQL, MS-SQLExperience building scalable and maintainable data intensive applicationsPipeline orchestration technology, such as Prefect, Luigi or AirflowBig data technologies, such as Hadoop & Spark, ideally including Python pandas and DaskCore CompetenciesKeen interest in data engineering and a “tinkering” mindsetDriven to continually learn about and incorporate new technologiesThrive in a self-driven environmentUnderstanding and integrating human and machine workflowsEnterprise Data Lake & Warehouse Modeling & DesignDocker, KubernetesCloud infrastructure, such as Azure, AWS, or Google CloudSome Full Stack Experience such as Flask and ReactDevelopment on LinuxOther Desirable AttributesDistributed systems – storage, compute & access patternsUnstructured Data extraction – pdf, scrapingGraph database, such as Neo4jElasticsearchServerless ArchitectureLocation – Spatial DataData catalog platforms, such as Amundsen This job description is a general listing of the required tasks and expectations of the position and in no way implies that the duties listed above are the employee’s only responsibilities. The employee is expected to perform other tasks, responsibilities and training as instructed by their supervisors. Duties and responsibilities may change at any time with or without notice.This position may require additional hours outside of the standard work schedule including occasional holiday, evening and/or weekend hours in order to meet deadlines or to accommodate customers.The employee will regularly be required to talk, hear, walk, use hands, kneel, crouch and lift up to 25 pounds. Reasonable accommodations may be made to enable individuals with disabilities to perform the essential functions.LightBox and all its holding companies are an equal opportunity/affirmative action employer. It is the policy of the LightBox and its holding companies to prohibit discrimination of any type and to afford equal employment opportunities to employees and applicants, without regard to race, color, religion, sex, national origin, age, disability, or veteran status.                 We thank all applicants for their interest; however, only those selected for an interview will be contacted. NO TELEPHONE CALLS OR AGENCY SOLICITATION PLEASE.