Artificial Intelligence Engineer

Artificial General Intelligence (AGI) Data Services is at the forefront of AI innovation, specializing

in the development and refinement of large language models (LLMs). Our mission is to create AI

systems that can understand and generate high-quality code, revolutionizing the software

development landscape.

Job Description: As an LLM Evaluation Expert specializing in Coding, you will play a crucial role in

assessing and improving our language models' coding capabilities. Your expertise will be

instrumental in evaluating LLM-generated code responses, making high-level judgments, and

setting the standard for what constitutes excellent AI-assisted coding.

Key Responsibilities:

• Critically analyze and evaluate code responses generated by our LLMs across various

programming languages and paradigms

• Exercise expert judgment to select the most appropriate and efficient code solutions from

multiple LLM-generated options

• Make informed decisions on behalf of our customers, ensuring that selected code meets

industry standards, best practices, and specific client needs

• Develop and write coding demonstrations to illustrate "what good looks like" in AI-generated

code, setting benchmarks for quality and efficiency

• Provide detailed feedback and explanations for your evaluations, helping to refine and

improve the LLM's understanding and output

• Collaborate with the AI research team to identify areas for improvement in the LLM's coding

capabilities

• Stay abreast of the latest developments in software engineering, coding standards, and AI to

ensure our evaluations remain cutting-edge

Required Qualifications:

• 3-5 years' development experience using Python or Java.

• Advanced degree in Computer Science, Software Engineering, or a related field

• Extensive experience in software development across multiple programming

languages(Python, Java) and paradigms

• Demonstrated ability to critically evaluate code quality, efficiency, and adherence to best

practices

• Strong analytical and decision-making skills, with the ability to make complex judgments

under ambiguous circumstances

• Excellent written and verbal communication skills, with the ability to explain technical

concepts clearly

• Experience with Docker

• Experience in technical writing, particularly in creating coding examples or tutorials

Preferred Qualifications:

• Previous experience working with or evaluating AI systems, particularly in the context of code

generation

• Familiarity with a wide range of software development methodologies and architectural

patterns

• Understanding of machine learning concepts, particularly as they apply to natural language

processing and code generation

• Experience in creating or contributing to coding standards or style guides

This role requires a unique blend of technical expertise, critical thinking, and communication

skills. You will be the bridge between advanced AI technology and practical, real-world coding

applications. Your work will directly influence the development of next-generation AI coding

assistants, shaping the future of software development.

If you're passionate about code quality, have a keen eye for detail, and are excited about the

potential of AI in software engineering, we encourage you to apply for this pivotal role at AGI

Data Services.