Azure Data Engineer Job at VDart Inc, Atlanta, GA

cjJtWEFzbGgxMnl5a0IzZ3FhK1A2UStEZUE9PQ==
  • VDart Inc
  • Atlanta, GA

Job Description

Job Title: Azure Data Engineer

Location: Atlanta, GA / Frisco, TX / Bellevue, WA

Duration: / Term: Contract

Experience Desired: 6+ Years

Job Description:

We are seeking a Data Engineer to build and maintain the foundational data pipelines that power our Customer Data Platform (CDP) - the authoritative source of truth for customer data across the entire US adult population.

This role focuses on developing reliable, scalable pipelines that ingest, transform, and deliver high-quality customer data in near-real-time. You will ensure that every customer record flowing through CDP is complete, accurate, and available when downstream systems - personalization engines, analytics, AI/ML models, and business intelligence - need it most.

You will work within a modern Azure/Databricks ecosystem, contributing directly to the data infrastructure that makes trusted, timely customer profiles possible at massive scale.

Job Responsibilities

  • Develop and maintain batch and streaming data pipelines using Azure Data Factory (ADF), Databricks, and Spark that ingest customer data from dozens of source systems into CDP
  • Build data ingestion frameworks from APIs, event streams (Event Hub/Kafka), and enterprise databases - ensuring every customer touchpoint feeds into the unified profile
  • Perform data transformation, cleansing, and enrichment using Python and Spark to maintain the accuracy and completeness that an authoritative source of truth demands
  • Implement data validation and quality checks at every pipeline stage - catching anomalies, duplicates, and schema drift before they erode trust in customer profiles
  • Work with ADLS, Snowflake, SQL Server, and Cosmos DB to store and process customer data across bronze, silver, and gold layers
  • Support data modeling and schema design under guidance from senior engineers, ensuring alignment with CDP's unified customer profile standards
  • Build curated data layers that serve analytics, reporting, personalization, and AI/ML use cases with production-ready, well-documented datasets
  • Ensure data pipelines meet SLA requirements for timeliness - customer data must be current and available when contextual experiences demand it
  • Support data pipelines used in identity resolution and AI/ML workflows by ensuring data readiness, consistency, and lineage traceability
  • Participate in job orchestration (Control-M), CI/CD pipelines, and monitoring - maintaining the operational discipline required for a platform serving the entire US adult population
  • Troubleshoot pipeline failures and resolve production issues with urgency, understanding that data delays directly impact customer experiences
  • Contribute to documentation, data dictionaries, and team knowledge sharing to strengthen the platform's institutional knowledge

Education and Work Experience

  • Bachelor's degree in Computer Science, Engineering, or related field
  • 3+ years of experience in data engineering or related roles
  • Experience working in cloud-based data platforms (Azure preferred)
  • Experience with Agile development methodologies

Technical Skills

  • Programming: Python (required), SQL, basic Spark (PySpark)
  • Azure stack: ADF, ADLS, Databricks (working knowledge)
  • Databases: Snowflake, SQL Server, Cosmos DB (basic to intermediate)
  • Solid understanding of ETL/ELT concepts and data pipeline design patterns
  • Hands-on knowledge with APIs and data integration patterns for diverse source systems
  • Experience with data quality checks, logging, monitoring, and alerting
  • Understanding of data lineage, metadata management, and how data supports downstream AI/ML workflows
  • Knowledge of CI/CD and orchestration tools like Control-M

Knowledge, Skills, and Abilities

  • Strong analytical and problem-solving skills with attention to data accuracy
  • Ability to work with large datasets and distributed systems at population scale
  • Eagerness to learn and adapt to emerging AI and data technologies
  • Understanding of why data quality and timeliness matter for customer-facing experiences
  • Good collaboration and communication skills across engineering and business teams

Key Skills:

Data Engineering, Azure, ADF, ADLS, Databricks, Python, AI/ML, ETL/ELT

Job Tags

Contract work, Work experience placement

Similar Jobs

Titan Placement Group

Pediatric Physician Assistant (PA) Job at Titan Placement Group

 ...Pediatric Physician Assistant (PA) Titan Placement Group invites you to explore an opportunity in Milwaukee, Wisconsin. Milwaukee is known for its vibrant arts scene, historic breweries, and scenic views along Lake Michigan. It offers an excellent mix of urban energy... 

Confidential

Aircraft Mechanic Job at Confidential

 ...Job Title: Aircraft Mechanic - THIS ROLE IS LOCATED IN MARANA, AZ (Travelers) Job Description We are seeking a skilled Aircraft Mechanic to join our certified repair station specializing in every aspect of aircraft maintenance. You will work on narrow and wide body... 

Leonardo Worldwide Corporation

Ricercatori/Ricercatrici - Quantum Technologies Labs Job at Leonardo Worldwide Corporation

 ...imprese. Innovazione, ricerca continua, digitalizzazione e sostenibilit sono i pilastri del suo business nel mondo. Allinterno dellArea Quantum Technologies Labs, stiamo ricercando Ricercatori/Ricercatrici per la nostra sede di Roma. La persona si occuper delle... 

Draken International Inc

Pilot Job at Draken International Inc

 ...the United States Military, Department of Defense and allied militaries globally. We are looking for experienced, dedicated Fighter Pilots to operate the largest fleet of commercial tactical military fighter jets as we challenge todays warriors for tomorrows fight.... 

Ascension

Family Medicine Physician Job at Ascension

 ...peer support, spiritual care and stress management resources Family support: parental leave, adoption assistance and family...  ...ll make an impact in this role Join an established team of 2 physician and 1 APPs Practice setting: outpatient Access to [Dedicated...