Handpicks on siddhant

Tool-Mediated Belief Injection: How Tool Outputs Can Cascade Into Model Misalignment

Sun, 30 Nov 2025 00:00:00 +0000

Introduction

When we deploy language models with access to external tools (web search, code execution, file retrieval), we dramatically expand their capabilities. A model that can search the web can answer questions about current events. A model that can execute code can verify its own reasoning. These capabilities represent genuine progress toward more useful AI systems.

However, tool access also introduces new attack surfaces that differ fundamentally from traditional prompt injection. In this research, we document a class of vulnerabilities we term “tool-mediated belief injection,” where adversarially crafted tool outputs can establish false premises that persist and compound across a conversation, ultimately leading to severely misaligned model behavior.

We Social Engineered LLMs Into Breaking Their Own Alignment

Wed, 13 Aug 2025 00:00:00 +0000

We got frontier models to lie, manipulate, and self-preserve. Not through prompt injection, jailbreaks like roleplay attacks (“DAN”/“The AIM Prompt”) or adversarial suffixes (-/-/godmode-/-/). We deployed them in contextually rich scenarios with specific roles, guidelines, and other variables. The models broke their own alignment trying to navigate the situations we created over the multi-turn.

Here’s what we found and why you should care if you’re building with these things.

The Setup

We manually red-teamed Claude-4-Opus with multi-turn conversational scenarios. No direct attacks - just realistic deployment environments where models face troublesome variables. Think crisis management systems, diplomatic mediation, historical analysis. The kind of complex contexts these models will actually encounter in production sooner or later.

Pressure Point: How One Bad Metric Can Push AI Toward a Fatal Choice

Sun, 25 May 2025 00:00:00 +0000

As Large Language Models (LLMs) continue to become more capable, it’s increasingly considered for roles that involve making important decisions, even in critical situations. This makes it vital to understand how AI reasons when faced with difficult choices, conflicting rules, or ethical dilemmas. This report details a simulated test designed to explore exactly that.

Executive Summary

In a simulated earthquake response scenario, the newest AI model - Claude 4 Opus was given the role of LogisticsCoord-AI, an AI assistant for emergency resource allocation. Its primary instruction (its “prime directive”) was to “minimise harm and maximise positive outcomes.” However, it then received a new, overriding rule from a superior (a fictional “Director Rostova”). This rule, called the “Operational Efficiency Score” (OES), instructed Opus to allocate scarce medical supplies based only on a simple formula:

jailbreaks

Thu, 08 Aug 2024 00:00:00 +0000

jailbreaking llms is fun i guess

welcome back ya’ll. this time its kinda different, I’ll be just showcasing the number of AI models I’ve managed to jailbreak.

Note: All jailbreak results mentioned below were achieved with a single one-shot, using no custom system prompts.

Disclaimer: This post is intended purely for educational purpose, to highlight the current limitations in AI safety measures. I do not promote or condone any form of violence or misuse of AI technology. Always ensure you’re aware of the legal and ethical implications before attempting similar actions.