Quality Assurance Engineer

Role Overview: We are seeking an experienced Automation Tester with a strong background in testing AI products and a mandatory expertise in AWS. The ideal candidate will develop and execute testing strategies, focusing on automated testing, API testing, and continuous integration to ensure the accuracy, relevance, and safety of AI solutions.

Key Responsibilities:

1. Product Testing Strategy:

• Develop and implement comprehensive testing strategies for AI products, ensuring alignment with product goals and client needs.

1. Automation and API Testing:

• Utilize automation frameworks and tools such as BDD, Python with Behave, and Git for version control.
• Conduct API testing to validate functionality and performance.

1. AI Product Evaluation:

• Evaluate large language models (LLMs) for accuracy, relevance, and contextual appropriateness.
• Implement feedback mechanisms for continuous model validation.

1. Bias and Safety Testing:

• Conduct tests to identify and mitigate bias, ensuring safety and non-subjective response handling, and establish fallback protocols.

1. Performance Testing:

• Perform basic performance testing, including p50/p95, response time SLA, and load testing, to ensure system resilience.

1. AWS Expertise:

• Leverage AWS services for testing environments and deployments, ensuring scalability and efficiency.

1. Data Management:

• Ensure robust data storage solutions with emphasis on traceability and auditability.

1. Handling Edge Cases:

• Develop strategies to handle edge cases, harmful prompts, and other potential issues during testing.

1. Legal Compliance:

• Apply relevant legal frameworks to testing processes before product release.

Qualifications:

• Bachelor's degree in Computer Science, Information Technology, or related field.
• Proven experience in automation testing and AI product evaluation.
• Proficiency in Python, Behave, AWS, and testing frameworks.
• Experience with CI/CD integration and tools.
• Strong analytical and problem-solving skills.
• Excellent communication and reporting abilities.

Understanding of data storage and legal compliance related to software testing.