Talent.com
Mercor
DevOps Engineer AI Model EvaluatorMercor • Berlin, Berlin, Germany
DevOps Engineer AI Model Evaluator

DevOps Engineer AI Model Evaluator

Mercor • Berlin, Berlin, Germany
Vor 30+ Tagen
Stellenbeschreibung

About the job

Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco our investors include Benchmark General Catalyst Peter Thiel Adam DAngelo Larry Summers and Jack Dorsey.

Position: DevOps / SRE / Cloud Engineer (Coding Agent Experience)
Type: Contract
Compensation: $85/hour
Location: Remote

Role Responsibilities

  • Use frontier AI coding agents to complete and evaluate complex infrastructure engineering tasks.
  • Review model-generated implementations involving cloud platforms Kubernetes CI/CD systems and infrastructure automation.
  • Identify bugs edge cases reliability issues and failure modes in model outputs.
  • Compare outputs from multiple frontier models to assess strengths and weaknesses.
  • Apply professional engineering judgment to realistic infrastructure engineering scenarios.

Qualifications

Must-Have

  • 2 years of professional DevOps SRE or Cloud Engineering experience.
  • Experience with AWS Azure GCP Kubernetes Terraform CI/CD pipelines or observability tooling.
  • Regular use of AI coding agents like Cursor Claude Code Codex Windsurf Gemini CLI or similar tools.
  • Ability to evaluate model-generated infrastructure and reliability engineering solutions.

Preferred

  • Experience supporting production-scale systems.

Compensation & Legal

  • $400 per accepted task.
  • Compensation is tied to accepted work.

Application Process (Takes 2030 mins to complete)

  • Upload resume
  • AI interview based on your resume
  • Submit form

Resources & Support

  • For details about the interview process and platform information please check:
  • For any help or support reach out to:

PS: Our team reviews applications daily. Please complete your AI interview and application steps to be considered for this opportunity.


Employment Type : Full Time
Experience: years
Vacancy: 1
Jobalert für diese Suche erstellen

DevOps Engineer AI Model Evaluator • Berlin, Berlin, Germany

Ähnliche Stellen

Generative AI Engineer

Few&Farberlin, Deutschland

Principal Consultant | Building Platform teams for start‑ups & scale‑ups @ Few&Far.A fast‑growing European startup is on a mission to bring.Backed by leading European VCs, they’re now expanding the... Mehr anzeigen

 • Gesponsert

Lead Analytics Engineer (f/m/d)_metrify

Enpalberlin, Deutschland

Germany’s largest competitive smart metering operator, and a new strategic spin-off initiative by Enpal.Are you ready for one of the great infrastructure challenges of the 2100s? To truly transform... Mehr anzeigen

 • Gesponsert

MLOps Engineer (m/w/d)

Deutsche Kreditbank Aktiengesellschaftberlin, Deutschland

Studium der (Wirtschafts-)Informatik, Data Science o.Mehrjährige Berufserfahrung im Bereich Machine Learning Engineering, ML Ops, Platform Engineering, Cloud Engineering oder DevOps.Sehr gute Pytho... Mehr anzeigen

 • Gesponsert

DevOps Engineer - AI Model Evaluator

MercorBerlin, Berlin, Germany
Homeoffice
Quick Apply

Headquartered in San Francisco, our investors include.DevOps / SRE / Cloud Engineer (Coding Agent Experience).Review model-generated implementations involving.Identify bugs, edge cases, reliability... Mehr anzeigen

AI Prompt Evaluator

Cross Border Talentsberlin, Deutschland

Bachelors degree, valid work permit for Germany.As soon as possible limited vacancies, apply fast!.Competitive salary + signing bonus (3,000 split over 3rd & 6th months).Apply here or send your CV ... Mehr anzeigen

 • Gesponsert

Applied AI / ML Lead Engineer

DAiNA Inc.berlin, Deutschland

Start Date: As soon as possible / by arrangement.DAiNA is a precision-oncology company focused on enabling personalized cancer treatment for individual patients.We combine comprehensive molecular t... Mehr anzeigen

 • Gesponsert

(Applied) AI Engineer

Qontextberlin, Deutschland

That takes people who are exceptionally talented, obsessed by the vision, and set on doing the best work of their lives.You’ll design and scale the backend that powers a company's brain, the logic ... Mehr anzeigen

 • Gesponsert

Systems & DevOps Lead Engineer (f/m/d)

Enwayberlin, Deutschland

Autonomous street cleaning is no longer a concept - it's on the road.At ENWAY, a Bucher Municipal company based in Berlin, we turn conventional street sweepers into self-driving machines through ad... Mehr anzeigen

 • Gesponsert

Deployment Engineer

telliberlin, Deutschland

Every 10 minutes, 18 million conversations happen between businesses and consumers around the world.For decades, the consumer side of those conversations sucked: long queues, robotic menus, repeate... Mehr anzeigen

 • Gesponsert

AI/ML Engineer - Fully Remote | Upto $85/hr

MercorBerlin, Berlin, Germany
Homeoffice
Quick Apply

Headquartered in San Francisco, our investors include.ML Engineer (Coding Agent Experience).Review model-generated implementations involving.Identify bugs, edge cases, performance issues, and failu... Mehr anzeigen

Lead Forward Deployed Engineer

Taktileberlin, Deutschland

Taktile is redefining how financial institutions use AI to make critical decisions, and we're growing fast.As a Lead Forward Deployed Engineer, you'll be at the heart of that transformation, owning... Mehr anzeigen

 • Gesponsert

AI Prompt Evaluator (Czech) | Essen, Germany

Cross Border TalentsBerlin, Germany
Quick Apply

Bachelors degree, valid work permit for Germany.As soon as possible limited vacancies, apply fast!.Competitive salary + signing bonus (3,000 split over 3rd & 6th months).Apply here or send your... Mehr anzeigen

Systems Engineer - AI Model Evaluator

MercorBerlin, Berlin, Germany
Homeoffice
Quick Apply

Headquartered in San Francisco, our investors include.Systems Engineer (Coding Agent Experience).Review model-generated code, architecture decisions, and systems implementations.Identify bugs, edge... Mehr anzeigen

DevOps Engineer (AI Exposure)

RepRiskberlin, Deutschland

RepRisk is the world’s most respected Data as a Service (DaaS) company for reputational risks and responsible business conduct.Our mission is to provide transparency on business conduct risks to dr... Mehr anzeigen

 • Gesponsert

DevOps Engineer (f/m/d) - Google Cloud Platform

smartclipberlin, Deutschland

We work remote by default – focused, efficient, and with full ownership.For larger features, architectural decisions, and real brainstorming sessions, we come together in Berlin or Cologne – fast, ... Mehr anzeigen

 • Gesponsert

AI Engineer (m/f/d)

Vectoryx USA Inc.berlin, Deutschland

We're building an AI-native claims operation.Our thesis: the moat isn't the model — it's the system around it that makes AI reliable enough to trust with real operational work.Building that system ... Mehr anzeigen

 • Gesponsert

(Senior) DevOps AI Engineer - Managed Services (all genders)

EXXETA AGberlin, Deutschland

Standorte: Berlin, Braunschweig, Dresden, Essen, Frankfurt am Main, Hamburg, Karlsruhe, Leipzig, Mannheim, München, Nürnberg, Stuttgart.Dein Herz schlägt für DevOps, Cloud und AI? Du möchtest moder... Mehr anzeigen

 • Gesponsert

Freelance DevOps Engineer

Avanai GmbHberlin, Deutschland

Avanai helps enterprises reimagine work with AI agents and automation that deliver measurable results, not surprises.We design, build, and run enterprise-grade AI agents and automated workflows tha... Mehr anzeigen

 • Gesponsert

Systems & DevOps Lead Engineer (f/m/d)

United States Digital Space LLCberlin, Deutschland

Autonomous street cleaning is no longer a concept - it's on the road.At the company, a Bucher Municipal company based in Berlin, we turn conventional street sweepers into self‑driving machines thro... Mehr anzeigen

 • Gesponsert

Senior AI DevOps / LLMOps

TechBiz Global GmbHBerlin, BE, DE

At TechBiz Global, we are providing recruitment service to our TOP clients from our portfolio.If you're looking for an exciting opportunity to grow in a innovative environment, this could be the pe... Mehr anzeigen