machine learning engineer for AI evaluation

Synthires develops AI research and evaluation projects focused on testing AI models on complex software engineering workflows.

• Design and develop Reinforcement Learning environments to evaluate AI models on realistic software engineering workflows;
• Create tasks involving bug fixing, feature implementation, codebase refactoring, and performance optimization;
• Develop reproducible environments with deterministic verification and golden reference solutions;
• Design workflows requiring AI agents to discover and reason over information using MCP tools and real MCP servers;
• Evaluate AI-generated software solutions for correctness, maintainability, performance, and scalability;
• Apply knowledge of algorithms, data structures, debugging, and systems engineering to create challenging evaluation tasks;
• Provide structured technical feedback to improve AI model reasoning and software engineering capabilities;
• Collaborate with technical teams to maintain consistent evaluation standards and high-quality deliverables.

• Strong proficiency in at least one of C++, Python, Java, Go, TypeScript, or Rust;
• Deep understanding of algorithms, data structures, and performance optimization;
• Demonstrated experience debugging complex software issues and delivering maintainable solutions;
• Strong background in feature development and codebase refactoring;
• Proven ability to optimize software for performance and scalability;
• Excellent analytical, problem-solving, written, and verbal communication skills;
• Strong attention to detail and ability to work independently;
• Experience collaborating effectively within cross-functional or remote engineering teams;
• Nice to have: Experience with large-scale or distributed codebases, familiarity with AI, machine learning, generative AI, or AI agent systems, experience with MCP, agentic workflows, or developer tools, background in rigorous code reviews, software testing, and engineering best practices, experience designing benchmarks, evaluation environments, or technical assessments, familiarity with DevOps, CI/CD, cloud infrastructure, or systems engineering.

• Hourly contract engagement;
• Part-time opportunity of approximately 15 hours per week;
• Competitive compensation of $60–$120/hour;
• Fully remote with flexible scheduling, including the option to choose preferred working days and hours;
• Output-based compensation may apply, with payment based on completed tasks meeting project specifications.