News on Medial

AI firms working on “constitutions” to keep AI from spewing toxic content

Arstechnica · 1y

Leading AI companies like Anthropic and Google DeepMind are developing "AI constitutions" to set values and principles for their models to follow and prevent abuses. As companies race to commercialize AI, concerns arise regarding the lack of "guardrails" to prevent AI systems from generating toxic speech and misinformation. The goal is to make these rules transparent and explicit, allowing users to understand and challenge the AI if it deviates from the principles. The current method of using reinforcement learning by human feedback is considered primitive and not accurate enough in refining AI responses.