Senior Product Engineer (Agent Training and Evals)

The role

As a Senior Product Engineer, you'll be one of the early engineers in our Agentic Training & Evals team.

You'll build the infrastructure that lets us evaluate AI agents, alongside real human expert work in safe and secure environments.

This is a genuine 0→1 role. There isn't a well-established playbook or existing architecture for you to simply maintain. You'll have significant influence and ownership over the technical direction of the team and the products we build.

You'll work closely with engineering, product and AI research to turn ambiguous problems into scalable systems, and you'll see the impact of what you build directly.

 

What you’ll bring to the role

Strong experience building scalable infrastructure or distributed systems in the cloud

Solid understanding of how systems are deployed, scaled and secured

Hands-on experience with evaluating AI models or agents

Interest in how AI agents are trained, evaluated and improved

Bonus: experience with image, video, or audio processing

Strong product mindset with a focus on delivering business and customer value

Comfort with ambiguity, and the ability to move quickly from idea to working system

Strong communication and collaboration skills

Python experience is useful, but not essential

 

What you’ll be doing in the role 

Solve hard problems around compute, orchestration, scaling, security and reliability

Help develop approaches for training, benchmarking and evaluating AI agents

Work with emerging agentic AI technologies and rapidly evolving models

Make pragmatic technical decisions in a highly ambiguous environment

Partner closely with product engineering and AI research to turn customer problems into working products

Drive direct impact in a fast-moving, high-ownership, cross-functional team

 

Key Technologies 

Cloud Platforms: Google Cloud Platform

Programming Languages: Python, JavaScript, and TypeScript

Frameworks: FastAPI, Vue.js Container-based and Serverless architectures

Databases: Postgres SQL

DevOps and Monitoring: GitHub Actions, Kubernetes, DataDog

 

Why Prolific is a great place to work

We've built a unique platform that connects researchers and companies with a global pool of participants, enabling the collection of high-quality, ethically sourced human behavioral data and feedback. This data is the cornerstone of developing more accurate, nuanced, and aligned AI systems.

We believe that the next leap in AI capabilities won't come solely from scaling existing models, but from integrating diverse human perspectives and behaviors into AI development. By providing this crucial human data infrastructure, Prolific is positioning itself at the forefront of the next wave of AI innovation – one that reflects the breath and the best of humanity.

Working for us will place you at the forefront of AI innovation, providing access to our unique human data platform and opportunities for groundbreaking research. Join us to enjoy a competitive salary, benefits, and remote working within our impactful, mission-driven culture.

 

Links to more information on Prolific

Benefits 

External Handbook  

Website 

Youtube  

 

Privacy Statement

By submitting your application, you agree that Prolific may collect your personal data for recruiting and global organisation planning. Prolific's Candidate Privacy Notice explains what personal information Prolific may process, where Prolific may process your personal information, its purposes for processing your personal information, and the rights you can exercise over Prolific use of your personal information.

Find Jobs in United Kingdom on Arbeitnow