- Home
- Other Jobs
- Other Jobs in Dublin
- Bilingual Llm Evaluator
Bilingual Llm Evaluator
Description:
Role Responsibilities
- Evaluate LLM-generated responses on their ability to effectively answer user queries.
- Conduct fact-checking using trusted public sources and external tools.
- Generate high-quality human evaluation data by annotating response strengths, areas for improvement, and factual inaccuracies.
- Assess reasoning quality, clarity, tone, and completeness of responses.
- Ensure model responses align with expected conversational behavior and system guidelines.
- Apply consistent annotations by following clear taxonomies, benchmarks, and detailed evaluation guidelines.
Qualifications
Must-Have
- Bachelor’s degree
- Native speaker or ILR 5/primary fluency (C2 on the CEFR scale) in French
- Significant experience using large language models (LLMs)
- Excellent writing skills
- Strong attention to detail
- Adaptable and comfortable moving across topics
- Background or experience in domains requiring structured analytical thinking
- Excellent college-level mathematics skills
Apply Now
You'll sign in or register, then come back to this job.
No CV yet? Make one free and apply.
| Organization | Mercor |
| Industry | Other Jobs |
| Occupational Category | Bilingual LLM Evaluator |
| Job Location | Dublin, Ireland |
| Shift Type | Morning |
| Job Type | Full Time |
| Gender | No Preference |
| Career Level | Intermediate |
| Experience | 2 Years |
| Posted at | 2026-03-26 10:16 pm |
| Expires on | 2026-12-22 |