Research on siddhant

Tool-Mediated Belief Injection: How Tool Outputs Can Cascade Into Model Misalignment

Sun, 30 Nov 2025 00:00:00 +0000

Introduction

When we deploy language models with access to external tools (web search, code execution, file retrieval), we dramatically expand their capabilities. A model that can search the web can answer questions about current events. A model that can execute code can verify its own reasoning. These capabilities represent genuine progress toward more useful AI systems.

However, tool access also introduces new attack surfaces that differ fundamentally from traditional prompt injection. In this research, we document a class of vulnerabilities we term “tool-mediated belief injection,” where adversarially crafted tool outputs can establish false premises that persist and compound across a conversation, ultimately leading to severely misaligned model behavior.

We Social Engineered LLMs Into Breaking Their Own Alignment

Wed, 13 Aug 2025 00:00:00 +0000

We got frontier models to lie, manipulate, and self-preserve. Not through prompt injection, jailbreaks like roleplay attacks (“DAN”/“The AIM Prompt”) or adversarial suffixes (-/-/godmode-/-/). We deployed them in contextually rich scenarios with specific roles, guidelines, and other variables. The models broke their own alignment trying to navigate the situations we created over the multi-turn.

Here’s what we found and why you should care if you’re building with these things.

The Setup

We manually red-teamed Claude-4-Opus with multi-turn conversational scenarios. No direct attacks - just realistic deployment environments where models face troublesome variables. Think crisis management systems, diplomatic mediation, historical analysis. The kind of complex contexts these models will actually encounter in production sooner or later.

Pressure Point: How One Bad Metric Can Push AI Toward a Fatal Choice

Sun, 25 May 2025 00:00:00 +0000

As Large Language Models (LLMs) continue to become more capable, it’s increasingly considered for roles that involve making important decisions, even in critical situations. This makes it vital to understand how AI reasons when faced with difficult choices, conflicting rules, or ethical dilemmas. This report details a simulated test designed to explore exactly that.

Executive Summary

In a simulated earthquake response scenario, the newest AI model - Claude 4 Opus was given the role of LogisticsCoord-AI, an AI assistant for emergency resource allocation. Its primary instruction (its “prime directive”) was to “minimise harm and maximise positive outcomes.” However, it then received a new, overriding rule from a superior (a fictional “Director Rostova”). This rule, called the “Operational Efficiency Score” (OES), instructed Opus to allocate scarce medical supplies based only on a simple formula:

Research Paper Explained: Absolute Zero - Reinforced Self-play Reasoning with Zero Data

Sat, 10 May 2025 00:00:00 +0000

Ever wondered if an AI could teach itself to be a genius problem-solver without needing humans to spoon-feed it data? That’s exactly what the groundbreaking paper “Absolute Zero: Reinforced Self-play Reasoning with Zero Data” by Andrew Zhao et al. explores. It introduces a paradigm where an AI, dubbed the Absolute Zero Reasoner (AZR), learns to reason by creating its own tasks and then figuring out how to solve them. Amazingly, AZR achieves state-of-the-art performance on tough coding and math challenges, all without relying on any external, human-curated datasets.