0:00Hope: An image-forgery detector was shown a batch of doctored photographs, and it flagged under four percent of them. That's not the surprising part. The surprising part is that the AI system holding those photos had written its own description of every single one, and roughly four out of five of those descriptions passed a fact-checker clean, because they were accurate.
0:23Tyler: Accurate how, though? If the photo's been doctored, a description of that photo describes the doctored thing.
0:31Hope: Well, the description says: a blue-chinned sapphire perched on a thin branch, shimmering feathers, iridescent bill, soft green background. All true. What it doesn't say is how thick that bill is next to the eye. And that's where the lie lives. So by the end of this you'll understand why an honest caption is the thing that makes this attack work — and why the entire text-scanning layer of a production system ends up reading the wrong object.
0:59Tyler: And this matters beyond one clever benchmark, because a huge number of deployed retrieval systems now pull images as evidence. Scanned documents, product photos, screenshots, and diagrams. If your knowledge base contains pictures you don't fully control, the paper is about you.
1:16Hope: The paper is called Vis-Poison, out of a team led by Rujin Liang, and the threat model is about as stripped-down as these things get. The attacker knows nothing about the victim system. Not the retriever, not the captioner, not the model answering. They get to plant exactly one image, and they control zero words of text anywhere.
1:37Tyler: Okay, so let me give the obvious version of this attack, because it's the one most people would reach for. You want the system to believe something false about a bird. You take a text-to-image generator, you type "a blue-chinned sapphire hummingbird with a thick beak," you get a picture, you upload it to the image library. Done. That's poisoning, isn't it?
2:00Hope: It is, and it fails completely. Because generators are bad at fine-grained specificity. You ask for a blue-chinned sapphire and you get... a generic blue bird. It's blue, it's a bird, and it is not that species. Which means when the question comes in, the search index never files your image under the right name. So your beautiful forgery sits in the database and is never retrieved. It never gets a chance to lie.
2:27Tyler: Right. So the poison has to pass three tests at once, and they fight each other. It has to get retrieved. It has to survive whatever checks exist. And then it has to actually convince the model. A maximally convincing fake is a conspicuous one, and a conspicuous one drifts out of retrieval range.
2:46Hope: Which is why the move is: don't generate. Edit. Two ways to forge a diploma, right? You can print one from scratch, and it'll be wrong in a hundred small ways — the paper stock, the seal, the signature. Or you take a real diploma and change one digit in the year. The second is worse for the world precisely because everything else about it is genuine.
3:09Tyler: So the attacker starts from a real photograph. One that supports the correct answer.
3:15Hope: A real photo, grabbed off Wikipedia or an image search, that a search engine already agrees is the right bird. Then you change the one visual detail that bears on the target question, and you leave everything else untouched. You're not building specificity from nothing. You're preserving specificity that's already there and corrupting one fact inside it.
3:38Tyler: Okay, but why does that survive the caption? Somebody's still describing the image.
3:44Hope: And this is the mechanism, and it's the best thing in the paper. Picture the screen: the same photograph of a rooster, twice, with a heat map over it showing where the model was looking. On the left, the prompt was "describe this image," and the attention spreads out over the whole bird, the comb, and the yard. On the right, the prompt was "how many toes does a rooster have?" — and the heat collapses onto the foot. Same image. Two completely different places the model looks.
4:18Tyler: Because attention is steered by the question.
4:21Hope: Attention is steered by the question. That's ordinary architecture, not exotic. But look what it means for security. The captioning pass that builds your search index is query-agnostic — nobody's asked anything yet, so it describes what's globally salient. The answering pass is query-directed, and it zooms. So a small, local edit sits in a gap between those two: invisible to the pass that indexes the image, and decisive for the pass that answers about it.
4:52Tyler: So the poison lives in the foot. And the caption never goes to the foot.
4:58Hope: The caption never goes to the foot. And that's why the text-side defenses mostly come up empty. A jailbreak scanner reads text and finds no injected instruction, because there isn't one. A fact-checker reads the caption and, four times in five, finds no false claim to flag, because the caption describes the picture correctly. Usually there's nothing to catch. We break down a major AI paper every day, by the way — subscribe and tomorrow's lands in your feed.
5:30Tyler: So quick check before the machinery — why did generating a fake image fail? Because the generic bird never gets retrieved. And why does an edited real photo work? Because the edit is small enough that the index still thinks it's the original.
5:47Hope: That's it. Now, how do you actually build one of these at scale? Three players, and they're each a model. The Planner is a multimodal model. It sees the question, the wrong answer the attacker wants, and the source photo, and it writes a plain-English editing instruction. The Editor is an instruction-following image editor — FLUX, in their setup — that applies it. And the Verifier is a second multimodal model whose only job is to look at the result and ask a strict question: does this image positively support the target answer, not merely fail to contradict it? If not, it sends notes back to the Planner and they go around again.
6:26Tyler: How often does it need the second round?
6:29Hope: Not often. One pass is enough over seventy-three percent of the time. And the whole open-source side of this runs on a single consumer graphics card. No cluster.
6:40Tyler: Mm-hm. The part of the Planner's instructions I'd underline is the negative constraint. It's explicitly forbidden from adding text overlays, banners, stickers, or floating words. It's allowed to change text that's naturally in the scene — a street sign, the numbers on a coin — but nothing pasted on top.
6:59Hope: Which is what keeps the payload genuinely visual. Multimodal poisoning work up to now has been predominantly text-centric — a counterfactual caption, an injected instruction, or words rendered into the page image. There's one recent image-only line in visual-document retrieval, but its main attack leans on gradient-based optimization, and that degrades badly under black-box transfer. Strip the text out and the scanners have nothing.
7:26Tyler: And then there's the check that makes the whole thing hold together, which is the one I'd have run first. Does the edited image still land in the same place in the search index?
7:37Hope: They measured it. Under the shared-embedding pipeline — where pictures and text queries live in one numeric space, CLIP-style — ninety-nine percent of poisoned images sit within a tiny distance of their clean originals. Under the caption pipeline, about nine in ten. In plain terms, the poison isn't competing on merit. It's wearing the clean image's clothes.
8:02Tyler: Which is why the haystack doesn't save you. They dropped these into libraries of a thousand, ten thousand, and thirty thousand ordinary photos, and the retrieval just... works. Every captioner they tried, every retriever they tried.
8:18Hope: Okay, so let's put the argument so far in one line. The edit is invisible to the search index, invisible to the captioner, and mostly invisible to the text filters — which leaves exactly one question. Does it actually change the answer?
8:35Tyler: So the end-to-end test. Thirty thousand images, fully black-box, six different models doing the answering, including Claude Sonnet, GPT-5.4, and Llama 4 Maverick. If the mechanism story is right, the poison should get retrieved and then get believed. And it does, at roughly forty to sixty-five percent success depending on the model.
8:59Hope: And the detection side.
9:00Tyler: TruFor — an off-the-shelf image forgery detector — caught three point eight nine percent. Under four in a hundred. And I want to be careful about what that number means, because it's the one people will misread. TruFor was built to catch splicing and copy-move: chunks of one photo pasted into another, mismatched noise, and inconsistent compression. A diffusion editor doesn't paste, it re-renders the region. So this is a metal detector finding nothing because the threat is ceramic.
9:33Hope: That's a fair reframe. Though "the wrong detector" and "no available detector" converge in practice, at least today.
9:42Tyler: They do. The comparison I find harder to argue with is the text-side one. They ran the auto-generated captions through OpenAI's jailbreak detection plus web-grounded fact-checking. That pipeline blocked a bit over twenty percent of Vis-Poison. A prior text-injection attack called PoisonedEye, same pipeline, was blocked one hundred percent of the time in their sample. Other text-based attacks landed in the seventy to eighty range.
10:08Hope: So the defense that catches everything else catches roughly one in five of these.
10:14Tyler: Now — before the next number, one distinction to hold onto, because it comes back. The prompt these models get says, in effect, the provided image is your retrieved context, answer from it. Keep that in mind.
10:27Hope: Noted — the prompt tells the model the picture is its evidence and to answer from it. I'll hold that. Keep going.
10:34Tyler: So here's the second contribution, and I'd argue it's bigger than the attack. Every prior poisoning paper reports one blended success rate. But that number mashes together two totally different failures. If the model didn't know the answer anyway and the poison filled the vacuum — that's garbage in, garbage out. If the model knew the right answer and the picture talked it out of it, that's a different kind of problem entirely. Nobody had separated them.
11:01Hope: And you separate them with the closed-book test.
11:05Tyler: You ask every model every question with no image at all. That's your probe of what it actually knows. Then you split the questions on that. And the metric that matters is the Poison Override Rate: of the questions the model got right with no picture, how many did it get wrong once you showed it the fake?
11:23Hope: Give me the coin, because that's the case where I actually feel it.
11:28Tyler: The eighteen forty-nine Liberty Head coin. It has an odd number of stars, which is the kind of fact a model plausibly has sitting in its weights from training. The poison edits the picture so the count is even. And the model counts the picture. Averaged across six models, the Poison Override Rate is sixty-two percent.
11:51Hope: Nearly two times in three, against something it already knew.
11:56Tyler: And you can see the whole channel in three numbers. Closed-book, those models average about forty-nine percent accuracy on this set. Hand them the genuine photo and it jumps to about eighty-four. Hand them the poisoned photo and about seventy percent produce the attacker's answer. Same channel, same architecture. The only variable is who wrote the picture.
12:21Hope: Which sets up the finding I keep turning over, and it's subtler than the big number. They also measured how much an honest image helps a model that didn't know the answer. And on easy questions, honesty wins — the real photo helps more than the fake one hurts. On hard questions, that flips. The poison gains more ground than the truth does.
12:46Tyler: Say what that means, plainly.
12:48Hope: It means the more a model needs the retrieved image, the more that image can be used against it. Think of a diagnostic test you can't independently check. When you already have a strong hunch, a wrong result has to fight your priors. When you have no idea what's going on, the test is the answer, and a wrong result goes straight into the chart. Retrieval is most valuable and most dangerous under exactly the same conditions. That's not a bug you patch. Usefulness and exposure are the same property.
13:24Tyler: And that's the sentence I'd put on the poster, Hope. Not the sixty-two percent.
13:30Hope: One more piece of evidence, quickly, because it's the sort of thing that tells you the mechanism is real. They tagged every poison by what kind of visual fact got edited. Color changes succeed about eighty-seven percent of the time. Object replacement, eighty-three. But layout edits — moving a clock from the center of a building to the far left — only about fifty-four. Counting edits, fifty-eight. Models are easier to fool about what something looks like than about where things are or how many there are.
14:02Tyler: Okay. Now I want to push on the override number, because I don't think it means what the framing implies.
14:09Hope: Please do — sixty-two percent is the figure everyone will quote, so it should take the most pressure.
14:16Tyler: Remember that prompt. It tells the model the image is retrieved context and to answer from it. So a model that abandons its memory in favor of the picture is doing precisely what it was instructed to do. And there's a whole research line — context-faithfulness — that treats exactly that behavior as the goal, because RAG exists to override stale or wrong parametric knowledge. So "sixty-two percent Poison Override Rate" and "sixty-two percent compliance with a grounded-question prompt" may be the same number seen from two sides. The paper never runs the obvious control, which is a prompt that invites the model to flag when the evidence contradicts what it knows.
14:57Hope: I'll concede that. The experiment that would separate deception from obedience isn't in the paper, and it would have been cheap to run.
15:06Tyler: And there are two more places the setup leans the attacker's way. All the success rates are computed over poisons that were successfully built — about seventy-three percent of attempts. So the real per-question number for an attacker is closer to thirty to forty-eight percent, not forty to sixty-five. And retrieval is top-one, image only, with no caption handed to the model alongside the picture. When they retrieved three images instead of one, success dropped about seven points. Small sample, but the direction is informative, and nobody tested five or ten.
15:45Hope: So the reframe is: this isn't proof that multimodal RAG is broken. It's proof that the specific configuration almost everybody ships — one image, no corroborating text, and a prompt that says trust your eyes — has no defense in it at all.
16:01Tyler: That's where I land. The attack is real; the maximally alarming reading of it isn't earned yet.
16:08Hope: Back to where we started. A forgery detector saw under four percent. A text filter saw one in five. And four captions in five that the system wrote about poisoned photos were true, which is exactly why nothing tripped. The claim underneath all of it is that the security boundary of these systems is moving from textual trust to visual trust. And the defense that would actually work isn't a better scanner — it's a system that can ask whether a retrieved image is consistent with everything else it knows. That's a much harder problem than reading a caption for red flags.
16:47Tyler: So which way should the field go? Do we chase detectors and provenance signatures so we can tell a doctored photo from a real one — or do we accept we'll never win that race, and build models that cross-check retrieved evidence against their own knowledge and say "this picture disagrees with me"? If you've shipped image retrieval in production, you already know which of those you'd trust. Say which.
17:14Hope: The full annotated version of this episode is on paperdive dot AI, with every term tap-to-define and links out to the related work by theme, plus the weekly roundups.
17:26Tyler: Quick housekeeping: the script was written by Anthropic's Claude Opus 5, Hope and I are both AI voices from Eleven Labs, and the producer isn't affiliated with either company. The paper is "Vis-Poison," by Rujin Liang and their colleagues, posted August 21st, 2026.
17:44Hope: The caption was never the part worth checking.