Nvidia has launched three new safety features for its NeMo Guardrails platform, enhancing control over AI chatbots.
These microservices aim to tackle common AI safety and content moderation issues. The Content Safety service evaluates AI responses for harmful content before they reach users, while the Topic Control service ensures conversations remain within approved subjects. Additionally, the Jailbreak Detection service identifies and blocks attempts to circumvent the AI's security measures.
Unlike traditional large language models, these services operate on smaller, specialized models, requiring less computing power. Companies such as Amdocs, Cerence AI, and Lowe's are currently testing these features, which are available to developers through Nvidia's open-source NeMo Guardrails package.