"Alerted.org

Summary:

Meta was built to help people connect and share, and over the last decade our tools have played a critical part in changing how people around the world communicate with one another. With over a billion people using the service and more than fifty offices around the globe, a career at Meta offers countless ways to make an impact in a fast growing organization.Meta is seeking LLM Evaluation Scientists to join our Meta Superintelligence Lab, focusing on the evaluation and benchmarking of large language models (LLMs) across language and multimodal domains. We are committed to advancing the field of artificial intelligence by developing rigorous methodologies and tools to assess and improve the capabilities, safety, and reliability of cutting-edge AI systems. We are looking for individuals passionate about LLM evaluation, benchmarking, prompt engineering, data analysis, and the development of robust evaluation frameworks. As an LLM Evaluation Scientist, you will have the opportunity to shape the future of AI by ensuring our models meet the highest standards of performance and safety at scale.Our internships are twelve (12) to sixteen (16), or twenty-four (24) weeks long and we have various start dates throughout the year.

Required Skills:

Research Scientist Intern, LLM Evaluation (PhD) Responsibilities:

1. Design, implement, and maintain comprehensive evaluation protocols for large language models, including both automated and human-in-the-loop assessments.

2. Develop and curate high-quality datasets and benchmarks to measure model performance, safety, fairness, and robustness across a variety of tasks and modalities.

3. Analyze model outputs to identify strengths, weaknesses, and failure modes, and provide actionable insights to research and engineering teams.

4. Collaborate with researchers, engineers, and cross-functional partners to define evaluation goals, communicate findings, and drive improvements in model quality.

5. Develop tools and infrastructure to streamline and scale evaluation processes, including dashboards, annotation platforms, and reporting systems.

6. Stay up-to-date with the latest research in LLM evaluation, benchmarking, and responsible AI, and incorporate best practices into Meta’s workflows.

7. Disseminate evaluation results through internal reports, presentations, and, when appropriate, external publications.

8. Contribute to the development of evaluation methodologies that can be applied to Meta product development and deployment.

Minimum Qualifications:

Minimum Qualifications:

9. Currently has or is in the process of obtaining a Ph.D. degree in Computer Science, Artificial Intelligence, Generative AI, or a relevant technical field

10. Must obtain work authorization in the country of employment at the time of hire and maintain ongoing work authorization during employment

11. Experience with Python, C++, C, Java or other related languages

12. Experience building systems based on machine learning and/or deep learning methods

Preferred Qualifications:

Preferred Qualifications:

13. Intent to return to the degree program after the completion of the internship/co-op

14. Proven track record of achieving significant results as demonstrated by grants, fellowships, patents, as well as first-authored publications at leading workshops or conferences such as NeurIPS, ICML, ICLR, ACL, EMNLP, CVPR, ICCV, ECCV, or similar

15. Experience working and communicating cross functionally in a team environment

16. Experience in advancing AI techniques, including core contributions to open source libraries and frameworks in computer vision

17. Publications or experience in machine learning, AI, computer vision, optimization, computer science, statistics, applied mathematics, or data science

18. Experience solving analytical problems using quantitative approaches

19. Experience setting up ML experiments and analyzing their results

20. Experience manipulating and analyzing complex, large scale, high-dimensionality data from varying sources

21. Experience in utilizing theoretical and empirical research to solve problems

22. Experience with deep learning frameworks

Public Compensation:

$7,650/month to $12,134/month + benefits

**Industry:** Internet

Equal Opportunity:

Meta is proud to be an Equal Employment Opportunity and Affirmative Action employer. We do not discriminate based upon race, religion, color, national origin, sex (including pregnancy, childbirth, or related medical conditions), sexual orientation, gender, gender identity, gender expression, transgender status, sexual stereotypes, age, status as a protected veteran, status as an individual with a disability, or other applicable legally protected characteristics. We also consider qualified applicants with criminal histories, consistent with applicable federal, state and local law. Meta participates in the E-Verify program in certain locations, as required by law. Please note that Meta may leverage artificial intelligence and machine learning technologies in connection with applications for employment.

Meta is committed to providing reasonable accommodations for candidates with disabilities in our recruiting process. If you need any assistance or accommodations due to a disability, please let us know at [email protected].