"Alerted.org

Job Description

Qualifications:

• 10+ years of experience as Data Engineer

• Experience developing data pipelines

• Data Engineering Certifications

o Cloud Data Engineering Certifications

o Cloud Architecture Certification

• Strong experience working in Google Cloud Platform (GCP)

o Development in Cloud Run/Cloud Function, Pub/Sub Integration

o BigQuery development/operations

o IAM, Cloud Logging, Monitoring principles, operations

Compensation:

$60-65HR

Exact compensation may vary based on several factors, including skills, experience, and education.

Benefit packages for this role will start on the 31st day of employment and include medical, dental, and vision insurance, as well as HSA, FSA, and DCFSA account options, and 401k retirement account access with employer matching. Employees in this role are also entitled to paid sick leave and/or other paid time off as provided by applicable law.

We are a company committed to creating diverse and inclusive environments where people can bring their full, authentic selves to work every day. We are an equal opportunity/affirmative action employer that believes everyone matters. Qualified candidates will receive consideration for employment regardless of their race, color, ethnicity, religion, sex (including pregnancy), sexual orientation, gender identity and expression, marital status, national origin, ancestry, genetic factors, age, disability, protected veteran status, military or uniformed service member status, or any other status or characteristic protected by applicable laws, regulations, and ordinances. If you need assistance and/or a reasonable accommodation due to a disability during the application or recruiting process, please send a request to [email protected] learn more about how we collect, keep, and process your private information, please review Insight Global's Workforce Privacy Policy: https://insightglobal.com/workforce-privacy-policy/.

Skills and Requirements

Day to Day:

The Data Engineer will join our automotive client’s team working in one of the world’s largest data lakes. This team’s current goal is to move their data from on-prem to GCP by utilizing Airflow and BigQuery. You will work with on-prem data sources (Hadoop, SQL Server, Teradata, etc.), understand the data model & business rules behind the data, and build data pipelines (with GCP) using Airflow for workflows and pipeline management. You will also partner with IT source teams to ingest data from various sources into a big data environment and streamline data ingestion processes. This data will be landed in GCP BigQuery.

Responsibilities:

• Design data models for data warehouse with standards for consistency and integrity

• Architect and develop data warehousing processes

• Develop and maintain data dictionaries/metadata management processes for source data schema

• Streamline data ingestion processes

• Develop scalable, re-usable and metadata-driven solutions for data landing and transformation

• Develop repeatable processes that accelerate delivery of user request fulfilment

Monitor data pipelines from sources to a big data environment