"Alerted.org

Overview

We’re looking for a hands-on engineer focused on quality, evaluation, and reliability of AI/ML systems, including generative AI. This role sits at the intersection of SDET, ML engineering, and model evaluation.

You’ll spend a significant portion of your time writing code to build automation and evaluation frameworks that validate how models behave in real-world conditions. While traditional testing is still part of the role, the primary focus is on model evaluation, safety, and reliability.

What You’ll Do

+ Build and maintain automated frameworks to evaluate ML and GenAI systems across data, model, and inference layers.

+ Define and execute model evaluation strategies using metrics such as accuracy, precision, recall, F1-score, latency, drift, and consistency.

+ Evaluate generative AI outputs for hallucinations, bias, safety, and adherence to guardrails.

+ Design and implement guardrails, red-teaming, and adversarial testing strategies for LLM-based systems.

+ Partner closely with data scientists, ML engineers, and product teams to integrate evaluation into model development and deployment workflows.

+ Embed evaluation and validation into CI/CD and MLOps pipelines to support continuous quality monitoring.

+ Document findings, surface risks, and help influence quality and reliability practices across the ML lifecycle.

What We’re Looking For

+ Experience as an SDET, ML Engineer, or Quality Engineer working on AI/ML systems, with a strong emphasis on validation and evaluation.

+ Strong programming skills in Python, with hands-on experience building automation or evaluation frameworks.

+ Practical experience evaluating ML models and/or LLMs, including working with performance metrics and behavioral testing.

+ Familiarity with model lifecycle concepts: data validation, training, deployment, inference, and monitoring.

+ Experience working in cloud environments (AWS, Azure, or GCP) and integrating with CI/CD or MLOps pipelines.

+ Comfort working with probabilistic systems where outcomes are measured, not binary.

+ Bachelor’s degree in Computer Science, Engineering, or equivalent practical experience.

Nice to Have

+ Experience with tools for model evaluation, monitoring, or explainability (e.g., MLflow, Weights & Biases, SHAP, LIME).

+ Exposure to GenAI safety, responsible AI practices, or model governance.

+ Background working in cross-functional teams with data science and product stakeholders.

What Makes HTC A Great Place To Build Your Future

HTC Global Services wants you to join our team. Come build new things with us and advance your career. At HTC Global, you’ll collaborate with experts, work alongside clients, and be part of high-performing teams driving success together. You’ll have long-term opportunities to grow your career and develop skills in the latest emerging technologies.

At HTC Global Services, our employees have access to a comprehensive benefits package. Benefits can include Group Health (Medical, Dental, and Vision), Paid Time Off, Paid Holidays, 401(k) matching, Group Life and Disability insurance, Professional Development opportunities, Wellness programs, and a variety of other perks.

Our success as a company is built on inclusion and diversity. HTC Global Services is committed to providing a workplace free from discrimination and harassment, where every employee is treated with dignity and respect. We celebrate differences and believe that diverse cultures, perspectives, and skills drive innovation and success. HTC is an Equal Opportunity Employer and a proud National Minority Supplier. We seek to empower each individual, fostering an environment where everyone feels valued, included, and respected.

#LI-SK11 #LI-Hybrid