Part-time • Mercor
Pay: $70 - $130 / hour
Role overview
This role focuses on reconstructing and annotating reasoning processes behind model-generated solutions to real-world software engineering tasks. You’ll work with SWE-bench–style problems derived from open-source repositories and analyze how an AI model approaches debugging, refactoring, and feature implementation.
Instead of writing production code from scratch, you’ll evaluate model trajectories and rebuild the step-by-step technical reasoning that would realistically lead to a correct solution. The work blends hands-on software engineering experience with structured analytical writing.
This role is suited for experienced engineers who are comfortable navigating real codebases, debugging complex issues, and clearly articulating how and why a solution works.
What you’ll actually be doing
- Review model-generated code trajectories on realistic software engineering tasks
- Analyze repository context, issues, failing tests, and existing implementation details
- Reconstruct step-by-step technical reasoning that explains how a developer would approach the problem
- Annotate decision points, trade-offs, debugging logic, and solution strategies
- Validate technical accuracy and ensure reasoning aligns with the actual code changes
- Work across tasks involving bug fixes, refactors, edge-case handling, and test alignment
Who this role is for
- Engineers with real-world experience debugging and modifying existing codebases
- Developers comfortable reading unfamiliar repositories and quickly building context
- Strong problem-solvers who can explain their technical thinking clearly in writing
- Professionals interested in applied AI research and model evaluation
- Engineers who value precision and structured analysis over rapid feature shipping
Who this role is likely NOT for
- Junior developers without substantial debugging experience
- Engineers who primarily build greenfield projects but rarely work in mature codebases
- Developers uncomfortable writing structured technical explanations
- Candidates without experience working with version control workflows
- Those looking for a purely implementation-focused coding role
Technical background
- 2+ years of professional software engineering experience
- Bachelor’s degree in Computer Science, Software Engineering, or related field (advanced degree preferred)
- Strong proficiency in at least one major language commonly used in open-source ecosystems (e.g., Python, JavaScript, TypeScript)
- Solid understanding of Git workflows, pull requests, and issue tracking
- Demonstrated experience debugging real production or open-source code
- Ability to produce technically accurate, well-structured written reasoning
Project scope
Opportunity for continued contributions based on quality and project needs
Short-term engagement with defined task batches
Expected commitment of approximately 15–25 hours per week, with flexibility for higher contribution depending on availability
Work is organized around discrete SWE-style tasks with measurable output expectations
