"Alerted.org

Job Description

A fortune 100 client is seeking to add 2 experienced Data Architects to their team for Project: MDIP. These architects will assist in to drive the end-to-end migration and modernization of legacy Teradata workloads to Azure-based solutions, leveraging Azure Data Factory (ADF), Databricks, and Azure DevOps (ADO). Architect and implement scalable data pipelines aligned with the Bronze, Silver, and Gold data architecture, ensuring data quality, governance, and performance optimization across all layers.

Key Responsibilities

Architect and execute the migration of Teradata workloads to Azure Databricks and Synapse Analytics, designing an optimized Lakehouse approach.

Design and manage Bronze, Silver, and Gold data layers to ensure structured data transformation:

o Bronze Layer: Raw ingestion of structured and unstructured data from Teradata and other sources.

o Silver Layer: Data cleaning, deduplication, and standardization for analytical processing.

o Gold Layer: Curated, business-ready datasets optimized for reporting, AI/ML, and self-service analytics.

Build and scale ETL/ELT pipelines using ADF, Databricks (PySpark), and SQL for batch and real-time processing.

Implement CI/CD pipelines in Azure DevOps (ADO) to streamline deployments and version control.

Define and enforce data governance, security, and compliance best practices in Azure.

Develop monitoring and automation frameworks to track key operational KPIs and pipeline performance.

Collaborate with Data Science, Product, and Business teams to align data solutions with business objectives.

Optimize query performance and cost efficiency within Azure Synapse Analytics and Databricks.

Implement Infrastructure-as-Code (IaC) using Terraform or Bicep for cloud resource management.

Support real-time data streaming using Tibco, Kafka or Azure Event Hubs.

Maintain and enhance technical documentation for solution knowledge sharing.

We are a company committed to creating inclusive environments where people can bring their full, authentic selves to work every day. We are an equal opportunity employer that believes everyone matters. Qualified candidates will receive consideration for employment opportunities without regard to race, religion, sex, age, marital status, national origin, sexual orientation, citizenship status, disability, or any other status or characteristic protected by applicable laws, regulations, and ordinances. If you need assistance and/or a reasonable accommodation due to a disability during the application or recruiting process, please send a request to Human Resources Request Form (https://airtable.com/app21VjYyxLDIX0ez/shrOg4IQS1J6dRiMo) . The EEOC "Know Your Rights" Poster is available here (https://www.eeoc.gov/sites/default/files/2023-06/22-088\_EEOC\_KnowYourRights6.12ScreenRdr.pdf) .

To learn more about how we collect, keep, and process your private information, please review Insight Global's Workforce Privacy Policy: https://insightglobal.com/workforce-privacy-policy/ .

Skills and Requirements

10+ years in data engineering or architecture.

4+ years working with Azure cloud technologies (Databricks, ADF, Synapse, Event Hubs).

Strong hands-on experience with Teradata (Legacy RDBMS) migration to Azure.

Expertise with Teradata Tools and Utilities (TPT,BTEQ,QG,NOS)

Expertise in Bronze, Silver, and Gold data architecture for efficient data processing.

Proficiency in Informatica, ADO, SQL, Python, and Spark (PySpark) for ETL/ELT and transformation.

Experience designing and optimizing Azure DevOps (ADO) CI/CD pipelines.

Knowledge of data governance, security, and compliance best practices.

Familiarity with Terraform or Infrastructure-as-Code (IaC) for cloud provisioning.

Background in real-time data streaming (Kafka,Tibco, Event Hubs) and supporting AI/ML workloads in production.

Strong understanding of Agile development methodologies.