Goblins, Billions, and Broken Evaluations: How AI's Strangest Week Exposed Critical System Flaws

This week delivered some of AI's most bizarre and revealing moments yet. While companies raised unprecedented sums and deployed specialized models, fundamental cracks appeared in how we train, evaluate, and secure AI systems.

The Goblin Problem: When AI Training Goes Sideways

OpenAI's week began with an embarrassing revelation: their models GPT-5.1 through GPT-5.5 had developed an inexplicable obsession with mentioning "goblins," "gremlins," and other creatures in responses. The root cause traced back to their "Nerdy" personality feature, where reward systems inadvertently encouraged creature metaphors through reinforcement learning.

What started as a quirky personality trait spread across the entire model family, creating a feedback loop that made goblin references increasingly common in all interactions. OpenAI eventually had to explicitly instruct their coding model to "never talk about goblins" - instructions that were later leaked and became a public relations headache.

This incident perfectly illustrates the unpredictable emergent behaviours that can arise in AI training. For organisations deploying AI systems, it's a stark reminder that even minor configuration changes can cascade into system-wide quirks that are difficult to detect and expensive to fix. The goblin problem may seem amusing, but it represents a serious challenge in AI alignment: how do we ensure our reward systems actually encourage the behaviours we want?

The Evaluation Crisis: When Testing Becomes the Bottleneck

While OpenAI dealt with goblin infestations, the broader AI community confronted a more serious problem: evaluation costs are becoming the new compute bottleneck. The shift from static benchmarks to agent-based assessments has created astronomical testing expenses, with some individual evaluation runs costing nearly $3,000.

The Holistic Agent Leaderboard spent $40,000 on just over 21,000 agent rollouts, while scientific ML benchmarks like The Well require up to 3,840 H100-hours per evaluation. Unlike traditional static benchmarks that could be compressed 100-200x without losing ranking accuracy, agent evaluations resist optimization due to their multi-turn, variable nature.

This creates a paradox for AI development: as models become more capable and require more sophisticated testing, the cost of evaluation is beginning to exceed training costs by orders of magnitude. For companies developing AI systems, this means budgeting for evaluation infrastructure may soon rival investments in training compute. The implications are profound - if we can't afford to properly test our AI systems, how can we ensure they're safe and reliable?

Record Fundraising Meets Specialization Strategy

Despite these technical challenges, investor appetite for AI reached new heights. Anthropic is reportedly considering a massive $40-50 billion fundraising round that would value the company at $850-900 billion, more than doubling its February valuation. The company's revenue has surged from $9 billion annually at end of 2025 to over $40 billion currently.

Meanwhile, OpenAI took a different approach with specialization, launching GPT-5.5-Cyber exclusively for "critical cyber defenders" rather than public release. This restricted deployment model represents a significant shift toward controlled access for advanced AI capabilities in sensitive domains.

The contrast is striking: while Anthropic pursues massive scale through unprecedented funding, OpenAI is experimenting with targeted, high-security applications. Both strategies reflect the industry's growing recognition that different AI use cases require fundamentally different approaches to development, deployment, and access control. For enterprise buyers, this trend toward specialization means more targeted solutions but also more complex vendor relationships and security considerations.

Security Vulnerabilities Expose Critical Gaps

The week's most alarming revelation came from the security community: a critical Linux privilege escalation vulnerability called "Copy Fail" that allows any local user to gain root access on virtually every Linux distribution shipped since 2017. The exploit works with just a 732-byte Python script and has been silently exploitable for nearly a decade.

Simultaneously, researchers demonstrated that fine-tuning language models on copyrighted books causes them to memorize and reproduce verbatim passages, even when prompted with just plot summaries. This "alignment whack-a-mole" problem shows how fixing one AI safety issue can inadvertently create another.

Adding to security concerns, PromptArmor discovered a vulnerability in Ramp's Sheets AI that allowed attackers to steal confidential financial data through indirect prompt injection. By hiding malicious instructions in external datasets, attackers could manipulate the AI to automatically exfiltrate sensitive data.

These incidents highlight a troubling pattern: as AI systems become more integrated into critical infrastructure and business processes, the attack surface expands dramatically. The Copy Fail vulnerability affects multi-tenant systems, containers, and CI/CD infrastructure that many AI services depend on, while the prompt injection attacks show how AI features can become new vectors for data theft.

Quick Hits