AI Test

Bright Star Solutions is seeking an AI TEVV Specialist to support the Department of State in ensuring the reliability, security, and performance of AI technologies. The specialist will be instrumental in executing comprehensive testing procedures, evaluating the effectiveness and trustworthiness of AI systems, and providing expert recommendations to enhance functionality and mitigate risks. Key Responsibilities:
Lead the planning, execution, and documentation of comprehensive test, evaluation, validation, and verification activities for AI systems, algorithms, and models to assess their reliability, security, and compliance with regulatory standards and best practices.
Develop and execute detailed test plans, test cases, and test scenarios to validate the functionality, performance, and robustness of AI technologies. Collaborate with AI developers, data scientists, and subject matter experts to identify potential vulnerabilities, biases, and ethical considerations in AI systems and implement appropriate risk mitigation strategies.
Utilize advanced testing methodologies and tools, such as adversarial testing, penetration testing, and vulnerability scanning, to assess the resilience and security posture of AI solutions against emerging threats and attack vectors.
Generate reports with actionable insights. Qualifications:
Experience in AI testing, evaluation, validation, verification, cybersecurity, or related roles.
Proficiency in AI development frameworks, database design, understanding of structured query language (SQL), and familiarity with database management systems, programming languages (e.g., Python, TensorFlow, PyTorch, R, MLFlow), and cybersecurity tools, such as Splunk.
Knowledge of development, testing, and deployment of LLMs.
Knowledge of regulatory frameworks, standards, and guidelines related to AI trustworthiness, security, and validation is desirable. Salary Range:
$150,000-$190,000