Skip to content
Jobport

Research Engineer, Benchmarks

Clera · Singapore 🇸🇬

On-siteFull-timeMid levelSoftware & ITMentions visa sponsorship

About the Role

Join a small, technical team building benchmarks to evaluate frontier AI agents on realistic, domain-specific workflows. You will own evaluation design and implementation, helping ensure benchmark results are rigorous, credible, and relevant to real-world performance.

What You'll Do

  • Design, implement, and maintain benchmarks for evaluating frontier agents on domain-specific tasks.

  • Work with subject-matter experts to translate real workflows into realistic tasks and evaluation criteria.

  • Build reliable infrastructure for running models and agents against benchmark tasks at scale.

  • Develop metrics and analyses to assess benchmark difficulty, reliability, and failure modes.

  • Validate whether benchmark results correlate with real-world evaluations and user needs.

  • Write clear documentation and reports for technical audiences.

What We're Looking For

  • Two or more years of experience in research engineering or machine learning engineering, building AI benchmarks, evaluation infrastructure, or agent environments.

  • Technical education or equivalent grounding, with strong first-principles reasoning about task design, scoring, and failure modes.

  • Proficiency in Python, Docker, and Linux, including experience operating infrastructure for AI models or agents.

  • Hands-on experience designing and running evaluations for AI agents or large language models, with relevant, focused experience in benchmarks or evaluation.

  • Experience developing metrics, statistical analyses, or validation studies, and translating workflows across domains into benchmark tasks.

  • Strong technical writing and communication skills. Published papers or technical writing on benchmarking, evaluation, or model failure modes are valued.

  • Experience with reinforcement learning pipelines, data generation, or agent evaluation is a plus.

  • Comfort working independently in an early-stage, fast-paced environment and navigating open-ended problems.

Compensation & Benefits

Base salary: $100,000 to $170,000 annually. Visa sponsorship is available.

Location

On-site in Singapore, Singapore.

Posted 8h ago (18 Nov 2026) · Source: Clera via Ashby

Similar jobs in Singapore

Clera · Singapore 🇸🇬

On-siteFull-timeVisa sponsorship

About the Role Join an early-stage AI infrastructure team focused on reinforcement learning environments and model evaluation. You will own automation for quality control of training data, helping ensure that data and evaluations remain reliable as usage scales. What You'll Do Bu

8h ago · Clera via Ashby

Clera · Singapore 🇸🇬

On-siteFull-timeVisa sponsorship

About the Role Join a small engineering team building infrastructure for AI training and evaluation systems. You will own the reliability, scalability, performance, and developer experience of core platform services that support demanding workloads. What You'll Do Own uptime, lat

8h ago · Clera via Ashby

Clera · Singapore 🇸🇬

On-siteFull-timeVisa sponsorship

About the Role Join a small engineering team building synthetic data systems for AI agents. You will turn domain-specific workflows into scalable training tasks, then develop ways to measure whether those tasks improve model capabilities. What You'll Do Build pipelines that trans

8h ago · Clera via Ashby

Pico · Singapore 🇸🇬

On-siteFull-time

Pico fuels the global capital markets community by providing exceptional market data services and customized managed infrastructure solutions. As financial industry experts at the center of markets and technology, we help our clients efficiently scale their business and quickly a

8h ago · Pico via Greenhouse