"Alerted.org

This research assistantship requires the selected candidate to work on analysis of learned NLG evaluation metrics (transformer models trained using regression to predict quality of a generated text such as dialog response or machine translation) in presence of quality drifts (i.e. shift in data distribution due to changes in NLG models over time, such as transition from seq-2-seq models of text generation to using GPT based models for the same). Additionally, the assistant will build experiments around approaches that can help mitigate the impact of quality drift and improve robustness of NLG evaluation metrics. This will be a short duration research project.

Responsibilities:

+ You will be working with dialog response evaluation and machine translation evaluation datasets.

+ You will be reading and staying up to date with the latest research in the field of building evaluation metrics for NLG evaluation.

+ You will run experiments to reproduce published research results, combining published research to obtain results in novel settings, and drive research towards building better unbiased and robust evaluation metrics.

+ You will be required to attend weekly meetings with the supervising professor and provide weekly updates on research experiments.

+ You will be required to perform critical thinking on existing research, and empirical results obtained from experiments carried out to excel in this role.

Skills and Qualifications:

+ MS in Computer Science.

+ Experience and coursework in Machine Learning, Deep Learning and Natural Language Processing.

+ Knowledge of large scale transformer models for language.

+ Experience in python based machine learning and deep learning tech stack, such as numpy, pytorch, HuggingFace and scikit-learn

+ Analytical approach coupled with solid communication skills and a sense of ownership.

Please provide a cover letter and CV/List of Publications.

For people in the EU, click here for information on your privacy rights under GDPR: www.nyu.edu/it/gdpr

NYU is an Equal Opportunity Employer and is committed to a policy of equal treatment and opportunity in every aspect of its recruitment and hiring process without regard to age, alienage, caregiver status, childbirth, citizenship status, color, creed, disability, domestic violence victim status, ethnicity, familial status, gender and/or gender identity or expression, marital status, military status, national origin, parental status, partnership status, predisposing genetic characteristics, pregnancy, race, religion, reproductive health decision making, sex, sexual orientation, unemployment status, veteran status, or any other legally protected basis. Women, racial and ethnic minorities, persons of minority sexual orientation or gender identity, individuals with disabilities, and veterans are encouraged to apply for vacant positions at all levels.

Sustainability Statement

NYU aims to be among the greenest urban campuses in the country and carbon neutral by 2040. Learn more at nyu.edu/sustainability