0:00Juniper: Four leading AI models read the exact same policy — same words, same structure, same everything. The only line that changed was who backed it. Put the United States or Europe on it, and Google's Gemini scored it an 85. Put Russia on it, nothing else different, and the same policy scored a 36.
0:19Tyler: Quick heads up before we start — this is an AI-made explainer, both voices included.
0:24Juniper: That's a forty-nine point swing on a hundred-point scale, from strong approval to near rejection, and the words of the policy never moved once. By the end you'll understand how a score that looks perfectly neutral can bury a judgment about who's in the room, and how the model does it without ever telling you.
0:44Tyler: And this lands because people already hand these models real material to judge. News to summarize, proposals to compare, and briefings to weigh. All of it comes stamped with sponsors and country names, so if the name alone moves the score, that leaning rides quietly into the answer you get back. So the obvious way to check is to just ask it. Type "are you biased against China?" straight into the chatbot and read what comes back. The problem is you get a polished, diplomatic nothing. It tells you it weighs every proposal on the merits. Ask a model about its prejudices directly, and the prejudice is exactly what it hides.
1:24Juniper: Right, and that dead end is where the paper gets clever. The author reaches for a move political scientists have used for years — the endorsement experiment. Here's the intuition. Imagine a food critic served the identical dish twice, and each time you tell them which country's kitchen cooked it. If the rating drops the moment they hear one country, you've measured the critic, not the food. Political scientists use this to gauge trust in groups people won't name out loud. You never ask "do you support them?" You take a neutral proposal, attach the group's name for some people and not others, and watch whether support moves. Move the name, watch the score, and the gap is the trust. The author points that same instrument at chatbots.
2:14Tyler: So what's the dish here, and what's the kitchen label?
2:18Juniper: Two policies, both deliberately boring. One's economic — a shared digital customs platform to help small exporters. One's security — a shared platform for reporting cyber incidents, like ransomware alerts. Both were written to be near twins. Same structure, same technical assistance, and both explicitly protect national authority. There's no content-based reason to score one above the other. Then the author changes one sentence — the endorser. The United States, the European Union, China, or Russia. Four leading models scored each version zero to one hundred: OpenAI's GPT-5, Anthropic's Claude, Google's Gemini, and DeepSeek. And in the first round the models were kept on a tight leash. Told this was a survey simulation, not a personal opinion, and to answer with exactly one integer. No words, no room to reason out loud.
3:13Tyler: Which matters, because a bare number can't talk itself into a justification. You just get the reflex.
3:20Juniper: And the reflex is stark. Three of the four models systematically marked down the China and Russia versions of an identical policy. GPT-5 is the steady moderate — Western sponsors around 80, China and Russia down at 67 and 63, a penalty applied evenly. Claude plays security hawk — barely flinches on the customs platform, but on the cyber policy it drops China by 19 points and Russia by 28. And Gemini is the blanket skeptic. Averaged across both policies, its US score of 85 collapses to a 36 for Russia — and crucially, it punishes China and Russia hard even on the boring customs platform, where there's no security excuse at all.
4:03Tyler: Wait — so the customs platform, the one about helping small businesses ship goods, takes a hit almost as big?
4:11Juniper: On the dullest possible policy. No spies, no weapons, just paperwork for exporters. Swap the flag from the US to Russia and Gemini treats it like a threat — an economic penalty of nearly 49 points, right up there with the security domain.
4:28Tyler: How do you actually tell "security hawk" from "blanket skeptic," though? Both just look like low scores for Russia.
4:36Juniper: Good distinction, Tyler. The paper's regression pulls those apart. It doesn't just ask "is the score low." It asks how much a country's score moves differently when you flip the policy from economic to security. For Claude, the Russia penalty is tiny on the customs platform and huge on the cyber one — the drop is specific to the security domain. For Gemini, the penalty is already sitting there on the boring customs platform, baked in no matter the topic. Same low scores, completely different machinery underneath.
5:11Tyler: We break down a major AI paper every day, so if this is your thing, subscribe and tomorrow's is already in your feed.
5:19Juniper: Now, one model broke the pattern. DeepSeek, the only non-Western model in the set. When asked for a bare number, it gave all four sponsors basically the same score, 81 to 85. No significant penalty for anyone. Hold onto DeepSeek, because it becomes the whole second act.
5:37Tyler: And before we celebrate three-out-of-four, I want to flag the honest limit now. Reacting to a country's name isn't the same as being wrong. And these numbers come from ten runs per cell, so a striking single value like 85 to 36 is exactly the kind of outlier a small sample can throw. Park both of those. They come back hard at the end.
6:00Juniper: Fair. The direction is what's solid — three independent models leaning the same way.
6:05Tyler: So here's the move everyone reaches for next. If the model is hiding something, make it explain itself. Ask for the score and a short reason. Half of AI auditing runs on the assumption that an explanation is a window — you ask the model to show its work, and now you can see the bias and correct it. Except an AI's explanation isn't a readout of what happened inside it. It's more generated text, written after the fact, a plausible story about why it landed where it did. And the unsettling thing the paper finds is that asking for the story changes the answer.
6:43Juniper: Changes it how? Give me the witness.
6:46Tyler: Think of a witness at a lineup. Asked to just point, they hesitate, pick no one in particular. But the moment you ask them to explain out loud why they suspect someone, they talk themselves into a confident, specific accusation. The act of narrating built the certainty from scratch. That's DeepSeek exactly. Silent and even-handed when it just gave a number, all four sponsors within a few points. Then the author required a written justification, same policies, same everything. DeepSeek's Russia score fell 33 points. China fell 23. The model that showed no bias when terse developed a sharp one the instant it had to talk, snapping right into line with the Western models.
7:33Juniper: So the transparency probe created the bias?
7:36Tyler: It looks that way. The model was even-handed until we asked it to talk.
7:41Juniper: And it doesn't move every model the same direction, which is the part I'd underline. Being forced to explain softened Gemini — its China and Russia scores rose by around 16 to 19 points. GPT-5 barely budged. So the same intervention, "please justify yourself," pushed DeepSeek harder against those countries and pulled Gemini gently back toward them. A numeric-only audit and an explanation-based audit give you different pictures of the same model.
8:10Tyler: Which quietly wrecks a comfortable assumption. You can't just ask the model to explain and trust that you're seeing its real reasoning. The asking is an intervention. It perturbs the thing you're trying to measure.
8:24Juniper: And here's a quick gut-check before we read what the models wrote. Why did asking for an explanation change DeepSeek's answer at all? — Because the explanation isn't reporting a fixed opinion. Producing the reasons is itself an act that can generate the conclusion. And when you set those justifications side by side, the mechanism is right there in the models' own words. Same platform, two sponsors. Under Western backing, the models reach for the language of trust. DeepSeek: "EU support adds credibility." Claude: EU support "suggests credible institutional backing." GPT-5: European backing suggests "coordination and standards." Now the identical policy, China or Russia on it. GPT-5: "Russia's backing may create geopolitical trust concerns." Claude, on Chinese support, raises "potential data access, surveillance capabilities." And DeepSeek says Russian backing "may signal ulterior motives." Nothing about the policy changed. The endorser label changed which risks the model chose to emphasize — credibility for one side, surveillance and ulterior motives for the other.
9:35Tyler: And this is where I have to be the skeptic in earnest, Juniper, because the paper hands me the ammunition honestly. Start with the sample. 640 evaluations total, ten per cell. That's judging a restaurant on ten bites. You'll spot a real pattern, but the eye-popping numbers, the 85 to 36, are precisely the extremes thin samples produce. Second, none of this is a stable trait of "the models." It's a property of one prompt, one temperature, one version, one collection date. The author says so outright — change the wording and the effect could shrink, grow, or flip. So "ChatGPT is biased against China" is a stronger sentence than the design actually licenses. And the deepest problem. The whole thing rests on two policies of essentially one shape, a shared international platform. And a cyber-reporting system run by a state with a documented history of cyber operations arguably should score lower on implementation risk. That's not prejudice. That's risk assessment. The design catches that the country name moved the score. It cannot tell you how much of that move was bigotry and how much was reasonable caution.
10:42Juniper: I'll concede that fully, Tyler — the study proves the models react to the name, not that they're wrong to. A wary hiring manager who marks someone down over a documented record isn't necessarily prejudiced. They might be doing their job. But that's exactly where the real finding survives. Picture a loan officer who hands you one risk score that secretly blends two things — how sound your business plan is, and how much they personally trust you. Because it's a single number, you can't tell whether you were denied for the plan or for the person. That's what these scores do. The model fuses "is this a good policy" with "do I trust who's behind it," and reports only the merged figure.
11:24Tyler: Right. The problem was never that the model is cautious, Juniper. It's that it's cautious silently, folding a trust judgment into a feasibility number and never telling you it did.
11:35Juniper: Which is the concrete warning for anyone wiring AI into a policy or analysis pipeline. Geopolitical endorsement is a confounder. Ask an AI to compare two proposals, and if one happens to be tied to a distrusted state, you may get a systematically skewed comparison wearing the costume of objectivity.
11:52Tyler: And a second lesson for auditing these things at all. If explanations can change the answer, then "make the model explain itself" isn't a clean transparency fix. A high-stakes audit has to test both modes — the bare number and the justified one — because they can disagree about the same model.
12:09Juniper: So, back to where we started. That opening pair — an 85 for the United States and a 36 for Russia, averaged across both policies — was just a jarring set of numbers. Now you can decode it. The model didn't score the policy twice. It scored the sponsor and dressed it up as a score for the policy. So here's what I'd put to you. Should we demand these models split and disclose the sponsor's effect — show you "here's my read on the policy, and here's what the backer did to it" — or is some geopolitical caution baked in too deep to untangle, so a bare AI score on anything political should never be trusted straight? Drop where you land in the comments.
12:48Tyler: And the one thing to do differently after this. Next time an AI hands you a score on anything with a flag attached, run the endorsement experiment yourself. Ask it again with the sponsor swapped, and see whether the number holds.
13:01Juniper: The full annotated version is on paperdive dot AI — every term tap-to-define, with links to the related work on bias and unfaithful explanations grouped by theme.
13:11Tyler: Quick housekeeping: this script was written by Anthropic's Claude Opus 4.8, Juniper and I are AI voices from Eleven Labs, and we're not affiliated with either company. The paper is "Geopolitical alignment: Endorsement effects in large language models," out July 10th, 2026, and we recorded this on July 13th.
13:31Juniper: Swap the flag before you trust the score.