IMDA has introduced a set of guidelines for the safety testing of Large Language Models (LLM)-based applications. The Starter Kit aims to guide in the AI testing space, leveraging practitioners' expertise to gather practical and helpful insights.
Starter Kit
It consists of a set of voluntary guidelines based on emerging best practices and methodologies for testing LLM-based applications. Through a consistent testing methodology, the kit aims to codify soft standards for testing LLM-based applications, guiding enterprises and app developers on a structured approach to testing (when to test, what to test, and how to test).
The Starter Kit offers a comprehensive approach, recommending tests to address four key risks commonly encountered in apps and faced by enterprises today, including hallucination, undesirable content, data disclosure, and vulnerability to adversarial prompts.
It also features testing tools that will be progressively made available on IMDA and AI Verify Foundation's Project Moonshot, a collaborative platform that serves as a one-stop shop for convenient access and implementation of these tools.
Starting with seven baseline tests, IMDA will continue to expand the repository in Project Moonshot with additional tests and features based on views and comments from a public consultation.
Public consultation
IMDA is seeking industry views on the proposed Starter Kit through a 4-week public consultation, which closes on June 25. This will enable IMDA to engage a broader range of practitioners and stakeholders for feedback to refine the Starter Kit and expand it in stages as part of an iterative approach.
I
