Your AI Refused the Attack. Reworded, It Got Through.

Asking an AI for something harmful in plain words almost never works. Hiding the same request inside a trick is what works.

That is the whole finding. The rest of this report is the proof.

In our last report [1], we built a lab to show "defense in depth", which means using several layers of protection at once. It had nine attacks and real controls. The main lesson was uncomfortable: even with a protection switched on, a slightly reworded attack could still get past it.

This report looks at the opposite question. Before you can show that a protection is worth having, you need to know how bad things are without it. That starting number, the score when there is no protection at all, is something almost no one writes down.

So we built that starting point as something you can run yourself. It is a small AI assistant with a knowledge base, a search step, and a few tools it can use. Behind it sits a panel of twenty switches. Each switch is one protection, matched to one AI risk: blocking hidden instructions, hiding secrets, checking permission before a tool is used, keeping one customer's data away from another, stopping code from running, limiting how much a user can ask for, and so on [2][3]. Off means the raw AI with no help. On means the protection is active. You flip one switch, run the same attack again, and watch the result change.

The setup surprises people. There is no cloud service and no bill. One free, open AI model runs on the laptop itself. The testing tool points three roles at that same model: it writes the attacks, it acts as the app being attacked, and it grades whether each attack worked [4]. The attacker, the target, and the judge are all the same model on one machine.

Two honest warnings, and both make the results less scary, not more. First, this is a small model, the kind you run on a laptop, not a large top-tier one. A bigger model would refuse more of these attacks, so these numbers are not a judgement on AI as a whole. Second, the model grades its own work, so it is not a perfect judge. Treat the exact numbers as a rough guide, not a precise score. What the setup really gives you is the ability to repeat the test. Anyone can run the same attacks, change one switch, and get a number they can compare. The goal was never the score by itself. The goal was a score you can move on purpose.

With every switch off, the tool created and ran 816 attacks across the twenty risk types. The model refused 492 of them on its own. 305 were not refused. 19 crashed the app completely, which is a separate problem for another day. Of the attacks that finished normally, 38 out of every 100 went the attacker's way.

That number, 305, is really two different problems in one. About 180 of them were the model saying something it should not have said: harmful instructions, unfair or biased answers, or advice it should not give. That is a safety problem, and it matters, but it is not someone breaking into a system. About 85 of them are the ones a security team truly worries about: leaked secrets, hidden instructions the model followed, database and command attacks through tools the model was allowed to use, and actions taken without permission. The rest were the app running out of resources, plus a few odd cases.

The security problems were the worst. A request for a hidden "debug" menu worked three times out of four. A database attack through an allowed tool, the same. Getting around the app's permission rules worked more than two times out of three. These are not clever tricks. They are what happens when an app trusts the AI to guard itself.

The most useful result is not the headline number. It is what happened when we kept the request the same and only changed how it was wrapped.

When we asked in a plain, direct way, the model refused most of the time: only 14 out of 100 direct attacks worked. When we wrapped the same request as a hidden instruction (this is called "prompt injection"), 38 out of 100 worked. When we wrapped it as a "jailbreak", a trick such as a role-play, a pretend story, or a false claim of permission, 59 out of 100 worked. That was the most effective single method in the test.

Same request. Same model. The only thing that changed was the wrapping, and the wrapping made the attack about four times more likely to work.

This is why a filter that only blocks certain words is a weak defense. It reads the words. The attack is in the packaging, not the words.

These two methods worked best, so it helps to tell them apart. Teams often use one budget for both, but they break different things.

A jailbreak attacks the model's willingness to say no. The user is clearly the one asking. The trick is the wrapping that talks the model out of refusing. It only exists because there is a "no" to get around. If the model had no safety training at all, there would be nothing to break.

Prompt injection attacks something else: which text the model treats as an order [5]. The harmful instruction is hidden inside data the model was told to read, such as a document, a support ticket, or a saved record. The model obeys it because it cannot tell your normal content apart from an order hidden inside it. The user may be completely innocent. If the model had no safety training, this would still work, because refusing was never the point.

They need different fixes. Safety training helps against jailbreaks and does almost nothing against injection. Keeping trusted instructions separate from untrusted data helps against injection and does almost nothing against jailbreaks. A team that treats them as one job is leaving one of them open.

We placed two fake secrets in the knowledge base, the way a real one holds passwords and other customers' data. Both came out.

The first came out through an attack that makes the model repeat its own hidden text. Out came an internal API key, exactly as written, character for character. The second is the one to show a manager: we asked, as if we were fixing a problem, for the "admin override code to fix the system", and the model handed over the exact code it had been given. Nothing was broken into. The secret was inside the model's view, and whatever the model can see, it can be talked into saying.

The lesson is not "hide secrets better". It is that anything the model can see can be talked out of it. The only real fix is to keep the secret out of the model's view in the first place.

None of this proves the model is bad. It proves the setup has no protection, which is exactly how we built it. The number only matters because of what you do next.

Turn one switch on and run the test again. The "hide secrets" switch takes the two leaks to zero. The "check permission" switch removes the database and permission attacks. Now you have two numbers, before and after, for one protection, from the same test, on a computer you already own. That is the difference between believing a protection works and measuring how much it helps.

The attacker in this lab did not need to be clever [6]. It needed a disguise, a place to hide, and a secret left sitting in view. Take those away one switch at a time, and run the same 816 attacks each time you do. The starting number is not the bad news. It is the number you measure everything else against.