DevOps Engineer | $17/hr Remote

Position: Terminal Bench Expert

Type: Contractor (Short-term 5weeks)

Compensation: $17/Hr

Location: Remote

Commitment: 20-40 hours/week (with 4 hours overlap with PST)

Role Responsibilities

• Design high-quality Terminal-Bench task ideas and specifications
• Develop complex tasks requiring reasoning, investigation, and debugging
• Write clear task descriptions, solution approaches, and verification logic
• Define deterministic, outcome-based evaluation criteria
• Identify realistic failure modes, edge cases, and operational constraints
• Create tasks that challenge AI systems while remaining solvable by experts
• Collaborate with reviewers to refine task quality and difficulty
• Contribute expertise across software engineering, infrastructure, workflows, automation, or related technical domains

Requirements

• Strong experience in software engineering or relevant technical domains
• Strong debugging, reasoning, and analytical skills
• Good understanding of system design, workflows, and dependencies
• Experience with production systems, pipelines, or large-scale workflows
• Strong technical writing and documentation skills
• Exposure to LLMs, agentic systems, or AI evaluation frameworks is preferred
• Familiarity with CLI tools, APIs, databases, automation workflows, or distributed systems
• Ability to work independently in a fast-paced remote environment
• Strong attention to detail and ability to follow project guidelines consistently
• Excellent communication and collaboration skills in remote work environments

Preferred Qualifications

• Experience with optimization techniques (linear programming, constraint satisfaction, scheduling algorithms)
• Background in operations research
• Experience with simulation or modeling tools
• Familiarity with AI planning systems or automated reasoning
• Experience with AI benchmarks (e.g., SWE-bench, Terminal-Bench)
• Hands-on experience with Docker

Application Process

• Apply or Easy Apply via LinkedIn to receive the application form by email.
• Fill out and submit the application form.
• Shortlisted candidates will receive the Vetsmith Assessment.
• Complete the 45-minute assessment, which evaluates technical reasoning, debugging, and problem-solving skills.
• Selected candidates will be contacted for pre-onboarding requirements.