PLEASE NOTE: You must take the Bilingual Competency interview in Bengali to be considered for this role. Location: Global. Type: Contract Work. Fluent Language Skills Required: Bengali (native fluency) and English (strong proficiency).Why this role mattersYour job is to assess Bengali AI‑generated responses and identify specific strengths and areas of improvement for these responses – your work will be used to create the "perfect AI‑generated response" at a later stage of this project. Note the analysis you create will be in English.What You'll DoConduct fact‑checking using trusted public sources and external toolsGenerate high‑quality human evaluation data by identifying response strengths, areas for improvement, and factual inaccuraciesAssess reasoning quality, clarity, tone, and completeness of responsesEnsure model responses align with expected conversational behavior and system guidelinesWho You AreYou hold a Bachelor's degreeYou are a native speaker in BengaliYou have significant experience using large language models (LLMs) and understand how and why people use themYou have excellent writing skills in English and can clearly articulate nuanced feedbackYou have strong attention to detail and consistently notice subtle issues others may overlookYou have a background or experience in domains requiring structured analytical thinking (e.g., research, policy, analytics, linguistics, engineering)Nice-to-Have SpecialtiesPrior experience with RLHF, model evaluation, or data annotation workExperience writing or editing high‑quality written contentExperience comparing multiple outputs and making fine‑grained qualitative judgmentsWhat Success Looks LikeYou identify factual inaccuracies, reasoning errors, and communication gaps in model responsesYou produce clear, consistent, and reproducible evaluation artifactsYour feedback leads to measurable improvements in response quality and user experience#J-18808-Ljbffr
Remote
Skills:
Analysis Skills, Artificial Intelligence (AI), Bengali Language, Communication Skills, Data Modeling, Detail Oriented, Editing, English Language, Fact Checking, Language Teaching, Linguistics, Modeling Languages, Quality Management, Training Data Sets, User Interface/Experience (UI/UX), Writing Skills
About the Company:
Mercor