• Data Engineer (Databricks)

    Insight Global (Newtown Square, PA)


    Job Description

    A healthcare payor is actively hiring for Data Engineers with strong Databricks expertise to help drive their data ingestion projects. This role will play a critical part in growing their Enterprise Data Office so the enterprise can leverage data to directly impact patient outcomes, operational efficiency, and data-driven decision-making. Key Responsibilities include:

    • Design, develop and implement end-to-end data solutions using Azure Databricks.

    • Convert legacy SQL workflows into modern Python-based pipelines.

    • Maintain and optimize high-volume data pipelines in production.

    • Write, test, and tune PySpark and SQL scripts for performance and scalability.

     

    PLEASE NOTE: We are looking for someone who values in person collaboration and is willing to come onsite weekly in Newtown Square, PA 19073.

     

    We are a company committed to creating diverse and inclusive environments where people can bring their full, authentic selves to work every day. We are an equal opportunity/affirmative action employer that believes everyone matters. Qualified candidates will receive consideration for employment regardless of their race, color, ethnicity, religion, sex (including pregnancy), sexual orientation, gender identity and expression, marital status, national origin, ancestry, genetic factors, age, disability, protected veteran status, military or uniformed service member status, or any other status or characteristic protected by applicable laws, regulations, and ordinances. If you need assistance and/or a reasonable accommodation due to a disability during the application or recruiting process, please send a request to [email protected] learn more about how we collect, keep, and process your private information, please review Insight Global's Workforce Privacy Policy: https://insightglobal.com/workforce-privacy-policy/.

     

    Skills and Requirements

     

    4+ years of experience as a Data Engineer working on data ingestion/integration related projects

     

    2+ years of experience building pipelines with Databricks

     

    3+ years of experience with Python for scripting, automation, data cleaning and transformation

     

    Understanding of PySpark to implement ETL pipelines

     

    3+ years of experience working with different data types (JSON, HL7, XML, CSV, etc.) - Azure Cloud environment

     

    - Experience working with HL7, FHIR and/or USCDI healthcare data standards