Freelance Business Operations Expert AI Evaluator

Uber AI Solutions is Uber's marketplace connecting skilled freelancers with Generative AI researchers and product teams working on cutting-edge AI systems. We partner with independent contractors to support large-scale model evaluation, alignment, and quality initiatives.

As a Freelance Business Operations Expert, you will pair genuine operational expertise with hands-on GenAI-evaluation experience to design challenging, realistic prompts backed by source documents such as process documentation, incident reports, operational performance data, and organizational policy documents. You will then probe where the model breaks, and encode expert judgment into precise, gradable rubrics and a canonical 'golden' answer. You own an end-to-end evaluation task from prompt design through reviewer sign-off.

This is hands-on operations work with an analytical core. You will produce operational deliverables as the output; what makes the work difficult is diagnosing causes and deriving conclusions from the provided source documents rather than restating them.

What you'll work on

• Apply your business operations expertise to help create and evaluate complex AI prompts and responses.
• Conceptualize and draft realistic, multi-step prompts that mirror real operational assignments and require causal analysis, comparison, and reconciliation across multiple source documents.
• Identify and document genuine model failures by writing specific, objective, and actionable explanations.
• Build comprehensive evaluation rubrics that trace each required input through dependent criteria and carefully separate extraction from analysis.
• Produce fully correct, deliverable-quality golden responses that satisfy all criteria using industry-standard formatting.
• Complete our onboarding process and strictly follow the disciplined authoring and review workflow for each task.
• Closely follow guidelines, run AI quality checks on your own work, and resolve any issues in the feedback loop before final submission.

Engagement details

• Location: Remote (United States)
• Engagement Type: 1099 Freelance / Independent Contractor, through the Uber AI Solutions platform
• Engagement Duration: 40 hours over 1-2 weeks

Who we're looking for

• 5+ years of experience in operational strategy, process improvement, or management consulting.
• MBA, MS in Management, or equivalent degree from an accredited institution.
• Demonstrated ability to read primary source documents (process documentation, incident reports, operational performance data, policy documents) and derive multi-step analytical conclusions from them.
• Familiar with root cause analysis methodologies, process mapping, and structured strategic frameworks.
• Comfortable working independently on detail-oriented tasks.
• Strong analytical and written communication skills.
• Prior hands-on experience with GenAI / LLM evaluation and prompt engineering, rubric or golden-answer authoring, data annotation, RLHF, red-teaming, or model quality assessment.
• Bonus: PMP (Project Management Professional) or Six Sigma Black Belt certification.

Why this matters

Your expertise will help improve how AI systems handle complex operational topics, making outputs more accurate, reliable, and better aligned with real-world business reasoning.