Job Title: Data Engineer

Company Name: Cofense

Job Details: RemoteFull,Time

Job Url: https://hiring.cafe/viewjob/5ly08wum0a59187z

Job Description: Posted 2d agoData Engineer@ CofenseView All JobsWebsiteUnited StatesRemoteFull TimeResponsibilities:Design data platform, Develop ETL pipelines, Collaborate with usersRequirements Summary:5+ years in data architecture/modeling; experience with SQL and NoSQL; ETL development; cloud DBs (Azure, AWS); Python; data pipelines; BI tools; REST APIs.Technical Tools Mentioned:Python, SQL, NoSQL, Azure, AWS, Power BI, Tableau, Splunk, DynamoDB, MongoDB, Hadoop, Spark, Kafka, Kinesis, SQS, Azure Queues, Bash
Reporting to the Manager, BI Development, the Data Engineer will be responsible for integrating enterprise wide data from Cofense’s Cyber Security based Applications, Micro-services and other disparate data sources.Essential Duties/ResponsibilitiesWork with Architects & Cloud Systems Engineers in designing Data Platform and ArchitectureSubstantial experience with SQL and no-SQL OLTP databases and OLAP data warehousing technologies, especially AWS AuroraExperience data modeling and building data pipelines for multi-product and/or multi-department organizationsDevelop Data Pipelines on Cloud Technologies like Azure/AWS with well-defined tool frameworksAble to develop ETL code to stream data from disparate (structured and semi-structured) SaaS product data stores to Data Lake/Data Warehouse using Python, Azure/AWS Data Lake servicesAbility to write complex SQL scripts and automate them using PythonDevelop test cases and unit tests for key implementations of Data Platform by adhering to software engineering best practices and standardsSecure data end-to-end by complying data privacy rules while developing processes to move data across Applications/Data Sources and Data Lake/Data Warehouse, as well as while delivering data through SQL clients and BI tools.Experience building adhoc reporting APIs and service layer on top of underlying OLTP and OLAP databasesHelp integrate Data Platform with BI tools like Power BI, Tableau, Splunk etc.Ability to develop and interpret Entity Relationship Diagrams (ERD) across data sets in relational database systems as well as non-relational Data storesAble to do Data Mining and Identify trends, patterns, anomalies in complex data sets across multiple data sources/systems and present results without ambiguityDevelop data transformations to generate Facts, Summaries, Key metrics by applying business rulesets and aggregations using Python, SQL and other transformation toolsAble to review current processes related to data ingestion, transformation and statistical analysis and re-engineer themCollaborate with business users across Cofense’s departments in defining requirements, prioritize project work and deliver them timelyOther duties as assignedKnowledge, Skills and Abilities RequiredExpertise in SQL skills for data transformations, statistical analysis, and troubleshooting across more than one Database Platforms (MySQL, PostgreSQL, Redshift, Azure SQL Warehouse etc.)Expert in writing complex SQL scripts and automate them using PythonKnowledge of Data management on NoSQL DBs like DynamoDB, Mongo, and know-how of Big Data tools Hadoop, Spark, Kafka/Kinesis/SQS/Azure Queues or other messaging tools is huge plusAnalytical skills, with good at finding data trends/outliers, anomalies, and articulate complex information or data points with Business Users, Management, and individualsEnthusiasm to work with lot of data across disparate data sources and DatabasesHas strong sense of engineering craftsmanship, takes pride in the code they write.Has a sense of intellectual curiosity and a burning desire to learn is self-driven, actively looks for ways to contribute, and knows how to get things doneIs deliriously customer-focused both internal and external customersSees big picture impact and relationships among and across work unitsIdentifies complex technical problems and tries to resolve with minimal helpEducation and/or Experience:Bachelor’s degree in Computer Science or Math, Data Analytics, Data Sciences, BI or demonstrated industry experience preferredOver 5 years of proven experience in data architecture, data modelling, and lifecycle managementHands-on experience with relational and cloud databases, including Azure SQL Database, Microsoft SQL Server, Amazon Aurora, and Amazon RedshiftStrong background in developing Python and data technologiesPractical experience designing and developing ETL data pipelines and applications using SQL and PythonStrong expertise in writing complex SQL queries for data transformation and automating processes using PythonExperience building and consuming RESTful APIs using Python librariesProficient in integrating data platforms with BI tools such as Power BI and Splunk, including dashboard and report developmentSolid experience working with Unix/Linux environments, including SSH tunnelling and writing/interpreting Bash scriptsAdvanced proficiency in Python 3, with hands-on experience using NumPy, SciPy, Scikit-learn, and PandasExperience leveraging APIs to extract data and load it into databases using Python - Disclaimer-The above statements are neither intended to be an all-inclusive list of the duties and responsibilities of the job described, nor are they intended to be a listing of all of the skills and abilities required to do the job. Rather, they are intended only to describe the general nature of the job. This job description is not a contract of employment, either express or implied. Employment with Cofense will be voluntarily entered into and your employment is considered at will. Cofense reserves the right to alter the job description at any time without notice.Cofense is committed to equal employment opportunity. We will not discriminate against employees or applicants for employment on any legally recognized basis [protected class] including, but not limited to: veteran status, uniform service member status, race, color, religion, sex (including pregnancy), gender identity, sexual orientation, national origin, age, physical or mental disability, marital status, genetic information or any other status or characteristic protected by applicable national, federal, state or local laws and ordinances. We adhere to these commitments in all aspects of employment, including recruitment, hiring, training, compensation, promotion, benefits, and discipline.