Talent.com
Mercor
Information Systems Expert - AI EvaluatorMercor • Berlin, Germany
Information Systems Expert - AI Evaluator

Information Systems Expert - AI Evaluator

Mercor • Berlin, Germany
Vor 30+ Tagen
Stellenbeschreibung
About the job Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors include Benchmark , General Catalyst , Peter Thiel , Adam D'Angelo , Larry Summers , and Jack Dorsey . Position: AI Model Evaluation Specialist Type: Contract Compensation: $40–$60/hour Commitment: 20 hours/week Role Responsibilities Write realistic prompts reflecting how professionals and consumers seek domain-specific guidance. Evaluate AI-generated responses for factual accuracy, regulatory or clinical correctness, and practical usefulness. Identify fabricated claims, incorrect references, or misleading reasoning across model outputs. Score and rank multiple model responses using structured rubrics across dimensions. Provide written justifications with specific evidence for each evaluation. Qualifications Must-Have Master’s degree or higher in Computer Science , Information Systems , or a relevant professional field. Professional experience applying domain expertise in a practitioner or advisory capacity. Familiarity with industry-specific standards, regulations, or clinical guidelines. Strong written communication and critical reasoning skills. Application Process (Takes 20–30 mins to complete) Submit your resume to begin. Complete the Model Response Evaluation assessment. Resources & Support For details about the interview process and platform information, please check: https://talent.docs.mercor.com/welcome/welcome For any help or support, reach out to: support@mercor.com PS: Our team reviews applications daily. Please complete your AI interview and application steps to be considered for this opportunity.
Jobalert für diese Suche erstellen

Information Systems Expert - AI Evaluator • Berlin, Germany

Ähnliche Stellen

System Engineer - AI Data Center Solutions (m/f/d)

Corning Inc.berlin, Deutschland

System Engineer - AI Data Center Solutions (m/f/d).The company built on breakthroughs.Corning is one of the world’s leading innovators in glass, ceramic, and materials science.From the depths of th... Mehr anzeigen

 • Gesponsert

Masterand – Innovative AI Data Solutions (m/w/d)

BearingPoint GmbHberlin, Deutschland

Als Fair Company bieten wir dir ein faires Gehalt.Du baust dir ein berufliches Netzwerk in einem international geprägten Umfeld auf – damit setzt du ein wichtiges Fundament für deinen Berufseinstie... Mehr anzeigen

 • Gesponsert

Expert for AI-Visibility & E-Commerce (m/w/d)

IT-Systemhaus der Bundesagentur für Arbeitberlin, Deutschland

Expert for AI-Visibility & E-Commerce (m/w/d).Die Printfamily verbindet spezialisierte Betriebe zu einem eingespielten System – ein Team, ein Plan, ein Ergebnis auf Niveau.Pinguin Druck, das Flaggs... Mehr anzeigen

 • Gesponsert

Senior Applied Scientist, AWS Developer Tools Science

Amazonberlin, Deutschland

Applied Scientist passionate about building services and tools for developers that leverage generative artificial intelligence and machine learning.You will be part of a team building Large Languag... Mehr anzeigen

 • Gesponsert

Enterprise AI Interface Specialist

MercorBerlin, Berlin, Germany
Homeoffice
Quick Apply

Headquartered in San Francisco, our investors include.Develop strategies to optimize the use of.Collaborate with teams to provide insights on.Direct experience using various company-specific.Applic... Mehr anzeigen

Applied Scientist - LLM, Alexa Conversational Modelling Intelligence

Amazonberlin, Deutschland

As an Applied Scientist II in the Alexa Conversational Modelling Intelligence team within Alexa AI, you will drive model post-training for Large Language Models that power Alexa+.You'll adopt and a... Mehr anzeigen

 • Gesponsert

AI Engineer (m/f/d)

Atlas Metricsberlin, Deutschland

Raisin is the world's leading platform for savings and investment products.Our vision is to offer savings and investments without barriers and thus open up the global 160 trillion euro market.Raisi... Mehr anzeigen

 • Gesponsert

AI Prompt Evaluator

Cross Border Talentsberlin, Deutschland

Bachelors degree, valid work permit for Germany.As soon as possible limited vacancies, apply fast!.Competitive salary + signing bonus (3,000 split over 3rd & 6th months).Apply here or send your CV ... Mehr anzeigen

 • Gesponsert

DevOps Engineer - AI Model Evaluator

MercorBerlin, Berlin, Germany
Homeoffice
Quick Apply

Headquartered in San Francisco, our investors include.DevOps / SRE / Cloud Engineer (Coding Agent Experience).Review model-generated implementations involving.Identify bugs, edge cases, reliability... Mehr anzeigen

Clinical Evaluator - Domain Expert

MercorBerlin, Berlin, Germany
Homeoffice
Quick Apply

Headquartered in San Francisco, our investors include.Clinical / biomedical / pharma Evaluator.Evaluate AI-generated artifacts against domain-specific quality rubrics.Identify factual, aesthetic, a... Mehr anzeigen

AI Systems User - Fully Remote | Upto $30/hr

MercorBerlin, Berlin, Germany
Homeoffice
Quick Apply

Headquartered in San Francisco, our investors include.Develop strategies to optimize the use of.Collaborate with teams to provide insights on.Direct experience using various company-specific.Applic... Mehr anzeigen

Principal AI Engineer - Conversational Banking

Rocketlistberlin, Deutschland

We are making a strategic investment in Artificial Intelligence and Generative AI to redefine how customers interact with N26.We are looking for a Principal AI Engineer to architect the next genera... Mehr anzeigen

 • Gesponsert

(Senior) Data & AI Engineer (m/f/d) - Tieto Tech Consulting

Tietoberlin, Deutschland

Do you turn raw data into real impact? Do you see patterns where others see chaos, and opportunities where others see complexity? As a (Senior) Data & AI Engineer, you know that data is more than n... Mehr anzeigen

 • Gesponsert

AI Prompt Evaluator (Czech) | Essen, Germany

Cross Border TalentsBerlin, Germany
Quick Apply

Bachelors degree, valid work permit for Germany.As soon as possible limited vacancies, apply fast!.Competitive salary + signing bonus (3,000 split over 3rd & 6th months).Apply here or send your... Mehr anzeigen

Croatian AI Evaluator Career Opportunity in Essen

Cross Border Talentsberlin, Deutschland

Are you fluent in Croatian and ready to play a key role in shaping tomorrows AI technologies?.Croatian-speaking AI Prompt Evaluator.In this position, you will work in a structured, international en... Mehr anzeigen

 • Gesponsert

AI Prompt Evaluator - Czech Market

FH Collective AcademyBerlin, Germany
Quick Apply

AI Prompt Evaluator Czech + English.We are seeking AI Prompt Evaluators fluent in Czech and English to contribute to the development and refinement of advanced AI technologies.Based on-site in Esse... Mehr anzeigen

Systems Engineer - AI Model Evaluator

MercorBerlin, Berlin, Germany
Homeoffice
Quick Apply

Headquartered in San Francisco, our investors include.Systems Engineer (Coding Agent Experience).Review model-generated code, architecture decisions, and systems implementations.Identify bugs, edge... Mehr anzeigen

AI Agent Evaluation Analyst (Freelance)

Mindriftworkfromhome, berlin, Deutschland

At Mindrift, innovation meets opportunity.We believe in using the power of collective human intelligence to ethically shape the future of AI.This opportunity is only for candidates currently residi... Mehr anzeigen

 • Gesponsert

Sr. Applied Scientist, Amazon Robotics

Amazonberlin, Deutschland

Are you interested in how to build AI reasoning systems that give provably correct answers? Are you excited by science at the interface of classical AI reasoning and Large Language Models (LLMs)? W... Mehr anzeigen

 • Gesponsert

AI Research Engineer

Nulegal GmbHberlin, Deutschland

Wir bauen die Infrastruktur, die KI in der Rechtsbranche tatsächlich belastbar macht, nicht als Demo, sondern in Produktion, mit messbarer juristischer Qualität.Dafür suchen wir Menschen, die tief ... Mehr anzeigen