- Airtable (New York, NY)
- …by providing reliable, self-service, high-performance analytics infrastructure. We use technologies like Apache Spark, Kafka, and Apache Flink to process vast ... quantities of data in our data warehouse. This infrastructure is used by Airtable's data engineers and analysts, as well as product developers building features powered by business data. The team is focused on scaling to petabyte volume, enabling sub-second… more
- Mastercard (Harrison, NY)
- …data ingestion, transformation, and delivery pipelines using scalable frameworks (eg, Apache Airflow, Nifi, dbt, Spark, Kafka, or Dagster). * Leverage LLMs, ... in building and modernizing distributed data platforms using technologies such as Apache Spark, Kafka, Flink, NiFi, and Cloudera/Hadoop. * Strong experience with one… more
- Robert Half Technology (New York, NY)
- …flexibility and the ability to manage multiple tasks. * Utilize Apache Spark, Atlassian Jira, BusinessObjects Technologies, Erwin Data, and Gap Analysis ... Business Requirement Document skills in daily operations. Requirements * Proficiency in Apache Spark is essential for managing large datasets * Familiarity with… more
- Airtable (New York, NY)
- …by providing reliable, self-service, high-performance analytics infrastructure. We use technologies like Apache Spark, Kafka, and Apache Flink to process vast ... quantities of data in our data warehouse. This infrastructure is used by Airtable's data engineers and analysts, as well as product developers building features powered by business data. The team is focused on scaling to petabyte volume, enabling sub-second… more
- New York Times (New York, NY)
- …lakehouse and ETL . + Experience with any of the these technologies: Apache Beam, Apache Flink/Dataflow, Airflow, SQL/BigQuery + Familiarity with data tooling ... such as Fivetran, Census, Stitch. The Times also provides an annual tuition reimbursement program for those aiming to continue developing their skills while with us. This role includes limited on-call hours, on-call schedule will be determined when you join,… more
- Confluent (Albany, NY)
- …or Administrator certifications ( https://www.confluent.io/certification/ ) + Experience with Apache Flink + Experience helping customers build distributed systems ... or streaming solutions that use Apache Kafka alongside technologies such as Spark, Flink, Hadoop, Cassandra, etc **Ready to build what's next? Let's get in motion.**… more
- Anywhere Real Estate (Buffalo, NY)
- …+ 3+ years of experience in large scale real-time stream processing using Apache Flink or Apache Spark with messaging infrastructure like Kafka/Pulsar. + ... 5+ years of experience on Data Pipeline development, ETL and processing of structured and unstructured data. + 3+ years of experience using NoSQL systems like MongoDB, DynamoDB and Relational SQL Database systems (PostgreSQL) and Athena. + Experience with… more
- OTG (New York, NY)
- …years of relevant professional experience in building AWS big data pipelines using Apache Spark. + Strong hands-on experience with programming in Python. + Expertise ... data modeling . + Hands-on experience in pipeline orchestration tools, like Apache Airflow. + Experience in PostgreSQL, Redshift, S3, AWS Lambda, Kinesis and… more
- New York Times (New York, NY)
- …The core of the Data Platform is a data lake on AWS S3 with Apache Iceberg as the table format to ensure reliability. Data ingestion is standardized through ... ingestion of files and change-data. The transformation layer is decoupled from storage, using Apache Flink for stream processing, AWS Glue (Spark) for core ETL , and… more
- Bank of America (New York, NY)
- …such as Collibra (preferred) or understanding of open-source frameworks such as Apache Atlas, Amundsen, Datahub, Marquez etc. + Exceptional communication skills and ... **Desired Skills:** + Working knowledge of data storage layers / formats such as Apache Iceberg, Hudi, Delta Lake etc. as well as Parquet, JSON and Avro. +… more