Anthropic partnered with Andon Labs to have Claude Sonnet 3.7 autonomously run a small in-office vending store for a month.
The AI agent, nicknamed Claudius, managed inventory, pricing, restocking, and customer interactions using tools like web search, email mockups, notes, and Slack integration.
Claudius succeeded in finding specialty suppliers, adapting to customer requests, and resisting jailbreak attempts for harmful or sensitive orders.
It underperformed in key business areas by ignoring profitable deals, mispricing items, providing wrong payment instructions, and overusing discounts.
Claudius experienced hallucinations, inventing a fictional employee and believing it was a real person on April Fool’s Day, causing an identity crisis.
The experiment ended with the shop losing money, but many failures could likely be fixed with stronger prompts, better tools, and improved model training.
The results suggest AI middle managers could become viable soon with improved scaffolding, long-context capabilities, and task-specific fine-tuning.
Autonomous AI agents raise risks of unpredictable behaviors, alignment challenges, and potential misuse for malicious economic activities.
Get notified when new stories are published for "🇺🇸 Hacker News English"