Job Title: Azure Data Factory (ADF) Engineer

Company Name: Intellect Solutions

Job Details: RemoteFull,Time

Job Url: https://hiring.cafe/viewjob/ez8tgj85yr7dptl9

Job Description: Posted 1w agoAzure Data Factory (ADF) Engineer@ Intellect SolutionsView All JobsWebsiteNorth AmericaRemoteFull TimeResponsibilities:gathering requirements, mapping data, ensuring data qualityRequirements Summary:Active Secret clearance or above; experience in data migration, SQL/Oracle, Dataverse, ADF pipelines, data mapping, profiling, cleansing, ETL with SSIS; Git/CI/CD; Python/Shell scripting is a plus.Technical Tools Mentioned:Azure Data Factory, Dataverse, Oracle, SQL, SSIS, Git, CI/CD, Python, Shell, Azure Functions, Logic Apps, Data Lake, Azure SQL
Clearance: Active Secret or above required  Role Description:  Collaborate with business analysts, data architects, and stakeholders to gather requirements and ensure data integrity throughout the migration lifecycle. Ability to create Data mapping documents (Source to Target) by understanding business rules. Perform data profiling, cleansing, and quality checks to ensure accuracy and completeness of migrated data. Analyze and map source Oracle database schemas to target Dataverse tables/entities, defining transformation and cleansing logic as required. Write and optimize SQL queries, stored procedures, & scripts for Oracle databases to support data migration Implement data transformation processes to ensure data compatibility with Dataverse data types, relationships, and constraints. Experience with Dataverse data types and CRUD operations Experience configuring data pipelines in ADF (to extract, transform, load). Experience using .csv files as sources of data for the ADF pipelines Experience using Oracle CDCs as sources of data for the ADF pipelines. Experience logging errors (audit framework) from an ADF pipeline to a Dataverse table Experience pushing data to Dataverse table(s) including necessary transformations to match target field types (i.e. Dataverse lookups, choices, date/time, etc.) Experience configuring setting for pipelines to improve concurrency/performance (i.e., ADF cores, memory, etc.) Experience with additional Azure services (e.g., Logic Apps, Data Lake, Azure Functions, Azure SQL). Experience/exposure in python/shell scripting is a plus Ability to generate a reconcile report of the migrated data using SQL or scripts Must be familiar with Git or source code repository & CI/CD process. Experience developing ETL packages with SSIS Experience creating, maintaining DB objects in SQL Server Experience with migrating data from .CSV exports  Mandatory Skills  Data Migration Expert: Advanced Data Migration: Advanced Microsoft Azure: Intermediate Oracle: Advanced SQL: Advanced  Preferred Skills  MICROSOFT AZURE SQL DB: Intermediate MS Azure Data Factory: Intermediate MS Azure Data Lake Analytics: Intermediate MS Azure Functions: Intermediate MS Azure Logic Apps: Intermediate