Role overview
This role focuses on designing and validating advanced Git workflow scenarios used to evaluate and train frontier AI systems. You’ll create realistic, high-difficulty version control environments that test how models handle complex repository states.
The work is technical and detail-oriented. It combines deep Git expertise with structured task design, automated validation, and rigorous quality assurance.
This role is suited for senior software engineers who understand Git beyond daily usage and can reason about history rewriting, repository state transitions, and failure recovery at a systems level.
What you’ll actually be doing
- Designing complex Git scenarios such as merge conflicts, interactive rebases, reflog recovery, bisect investigations, and branch manipulation cases
- Creating deterministic task prompts with clearly defined success criteria
- Writing automated test suites (e.g., using pytest) to validate repository state, commit history, and workflow correctness
- Developing reference (“oracle”) solutions using reproducible Git command sequences
- Defining grading rubrics that distinguish objective checks from qualitative evaluation
- Performing peer QA reviews on technical tasks to ensure consistency and technical accuracy
- Documenting workflows and ensuring artifacts meet strict technical standards
Who this role is for
- Engineers with 3+ years of hands-on Git usage in production environments
- Developers who understand Git internals, not just surface-level commands
- Professionals comfortable designing technical challenges or assessment-style tasks
- Engineers who enjoy precision, edge cases, and deterministic validation
- Contributors who can write clear, unambiguous technical specifications
Who this role is likely NOT for
- Developers who primarily use Git through GUI tools without understanding underlying mechanics
- Junior engineers without experience resolving complex merge or history issues
- Candidates unfamiliar with rebasing, reflog recovery, bisect workflows, or advanced branching strategies
- Professionals who prefer product feature work over structured evaluation or benchmarking tasks
- Engineers who struggle with precise documentation and test-driven validation
Technical background
- 3+ years of professional software engineering experience
- Deep knowledge of Git internals and advanced workflows (merge strategies, interactive rebase, reflog, bisect, history rewriting, recovery techniques)
- Proficiency in Python and shell scripting for automation and test development
- Experience building deterministic test frameworks or technical assessments
- Strong understanding of repository state validation and reproducible environments
- Familiarity with Docker and containerized development workflows preferred
Project scope
Flexible scheduling with output measured by technical quality and accuracy
Project-based engagement with potential for continued contributions based on quality and performance
Work centers on creating, validating, and refining advanced Git evaluation tasks
Structured documentation, tooling, and curated repositories are provided
Contributions are reviewed under a defined QA and rubric-based evaluation framework
