Why Coding Agents Lose Count After Fifty Items

0:00Bella: Today is November 22, 2026, and this is AI Papers: A Deep Dive. We walk through new AI research and what you can do with it. I'm here with Finn, and we're covering the research published since our last episode. Everything we looked at was posted or updated between November 17 and November 21, 2026. In that period, we went through 481 new and updated research papers and 88 new blog and release posts. From all of those, we picked four papers for you.

0:30Finn: One idea runs through all four papers today, and it is this. When a coding agent works something out but doesn't write it down, it often loses it. The clearest example is a counting test. A typical model can read a very long document, and yet it loses count of identical items after about fifty. We'll come back to that test in a few minutes. Okay, so here's what you're going to learn today. First, you'll get a five-minute test that you can run on any model. Second, you'll see why a coding agent can pass its tests and still break the rules you gave it. And third, you'll get two one-line prompt changes to try this week.

1:10Bella: Before we get to the counting test, it helps to understand what a coding harness does around a model. A coding harness is the program that sits between you and the model, and Claude Code is one example. Here's an analogy. Imagine a brilliant consultant who has complete amnesia between meetings. Before every meeting, an assistant hands the consultant one folder. It holds the original brief, the notes from every earlier meeting, and the results of every errand. The consultant reads the folder, says one thing or asks for one errand, and then forgets everything.

1:46Finn: In that picture, the assistant is the harness, and the consultant is the model. The folder is the context, which is all the text the model gets on that turn. The errands are tool calls, such as reading a file or running the tests. Here's a made-up example. You type, "Fix the failing test in the parser file." On turn one, the harness sends the model the system prompt, your sentence, and a list of tools. The system prompt gives the model its general instructions. The model asks to read the parser file, so the harness reads it and adds its contents to the folder.

2:22Bella: On turn two, the harness sends the whole folder again, from the top, and now the file is in it. The model asks to run the tests. So the harness runs them, adds the output, and sends everything a third time. In this made-up example, the folder might now hold eight thousand tokens. Tokens are the small chunks of text that a model reads. The model has gone through the folder from the top three times. Nothing carried over by itself. Only the written record carried over.

2:52Finn: But the analogy has one limit that matters today. Inside a single meeting, the consultant does hold things in their head that never reach the folder. These might be a half-finished count or a rule they noticed. So the question for the rest of the episode is what happens to the things the model works out but never writes down.

3:14Bella: The first paper measures that directly. It's a preprint, which means it hasn't necessarily been peer reviewed. It comes from Tianxiang Dai and Jonathan Fan in electrical engineering at Stanford. They wanted the simplest possible test of holding a running quantity, so they chose counting identical items. They give the model a long list of the same letter, the letter A, repeated over and over with commas in between, and they ask how many items there are. Every item adds one to the total, but no item records the total.

3:47Finn: Imagine counting marbles as they drop into a bucket in a dark room, with nothing to write on. Every marble looks the same, so there's no landmark. The only thing carrying the count is the number in your head, and at some point you lose track. The team ran this test on 205 models, making the list longer step by step. Every model they could measure eventually lost count, and the median model lost it at about fifty items.

4:14Bella: Now compare that with the context window, which is the total amount of text a model can take in at once. The median model stops counting after using less than a twentieth of one percent of its advertised window. So a model can read an enormous document and still fail to keep a simple tally across it. When it fails, it still answers with confidence, and the wrong answers are often round values.

4:40Finn: You may have heard that models miscount letters because of tokenization. That's the step that chops text into chunks. But it isn't the explanation here, because the items were cleanly separated.

4:53Bella: The part I found most useful concerns reasoning mode, which lets a model write out private working notes before it answers. The team compared the same models with reasoning on and off. The median gain in the counting limit was zero. But on a different task, reasoning improved performance severalfold. That task involved tracking objects through a written history of swaps.

5:17Finn: The difference is the written record. A history of swaps is text that the model can go back and reread, step by step. But a row of identical items gives it nothing to attach a note to.

5:29Bella: So here's what you can do with this. Paste a long list of the letter A, separated by commas, into your model at work and into a frontier model at home. Ask for the exact count, and make the list longer until the answer is wrong most of the time. That length is a cheap fingerprint of the model your harness is talking to. Then change your prompts so that the agent never carries a tally silently. If it scans many files for a deprecated call, have it write one labeled line per file, with the file name and a yes or no.

6:02Finn: My verdict is that the measurement is solid. The study is large and reproducible, and the logs and code are public. But the link from counting to long agent work is still a hypothesis, and the authors say so plainly. They also report that they found no prompt that raises the counting limit, so writing the state down is the remedy we have. So a count held in the head fades. The next paper asks about something else the agent is supposed to hold: the rules you gave it. It's a preprint that introduces a benchmark called ANCHOR. It comes from a team at several universities, including Southern University of Science and Technology and Nanjing University. Most coding benchmarks grade only the final code change against the tests. ANCHOR records the agent's whole run and grades it against rules a developer would attach. These include "only touch this directory", "reproduce the bug before you fix it", and "report the commands you ran".

7:02Bella: Think of hiring a contractor to fix a leaking pipe. You tell the contractor to work only in the bathroom, photograph the leak before touching anything, and hand you a list of what was done. An inspector who only checks that the leak has stopped will pass the job. That's true even if the contractor walked through your bedroom and gave you no list. ANCHOR is the inspector who follows the contractor around all day.

7:29Finn: The strongest system they measured was Claude Opus 4.8 running inside a scaffold called Cline. A scaffold is another word for the harness around the model. Even that system broke at least one stated rule on 81 percent of the tasks. The weakest areas were reporting what was done and following rules about order. The agents often did both required actions, but in the wrong order. Adding the rules barely changed whether the bug got fixed. So a benchmark that grades only the final code change would never reveal any of this. The failure analysis suggests an explanation, but it has a big caveat. The authors had a language model label each failure after the fact. The paper is explicit that those labels are descriptive hypotheses. They aren't observations of the agent's internal state or measurements of what caused the failure. With that said, the largest group, about thirty-eight percent, was labeled as cases where the agent never registered a rule at all. Only a small share was labeled as true limits of capability. Compliance also fell as more rules were added to one task. If those labels are right, how you present the rules is worth testing. But the paper never tests that as a remedy. ANCHOR uses its checklist to measure compliance, not to improve it. So try this on your own setup: give just a few rules per task, and write them as a numbered checklist. At the end, ask the agent to answer each item with yes or no, plus file paths and exact commands as evidence. Write ordering rules explicitly, such as "do not edit any file before you have reproduced the failure".

9:10Bella: I rate this one as solid as well, because many agent setups ran the same tasks. One caution is that a language model acts as a judge for part of the grading. The authors show that this judge is too lenient on ordering. Now, reporting was the weakest category in ANCHOR, and the third paper explains one reason why. It comes from the University of Wisconsin-Madison with William and Mary. It was published in the peer-reviewed journal Transactions on Machine Learning Research. The title says it well: "Thinking Is Not Telling". The team studied customer-service agents that have a think tool. A think tool is a private scratchpad where the agent writes notes into its own context, and the user never gets those notes.

9:56Finn: You'd expect that forcing the agent to think before every step would help. But across several models and scenarios, it didn't reliably help, and it often hurt. For GPT-4.1 on a retail benchmark, a strict consistency score fell by about eleven points.

10:12Bella: The reason is the interesting part. In one case, a user wanted to cancel a single item from an order, but the system could only cancel the whole order. The agent without the think tool said so, and the user chose to skip the cancellation. The agent with forced thinking wrote private notes about the request, but its reply to the user left the limit out. It simply asked the user to confirm, and the user said yes.

10:39Finn: It's like a travel agent who reads the fine print and then only asks, "Shall I go ahead?" Careful work behind the desk doesn't help you if the sentence that matters never crosses the counter. The fix the authors tested is one sentence in the system prompt. Before asking you to confirm any action, the agent must state the current state, the rules or limits that apply, the options, and the consequences.

11:05Bella: So the lesson isn't to turn thinking off. The deliberation was fine, but the telling was missing. I'd call the main result solid, with one limit. The authors replayed failed conversations with the missing information added, but the final recovery gains were small and inconsistent. For you, the step is simple. Add that disclosure line to your agent's system prompt, and run a few of your usual tasks with and without it.

11:32Finn: The last paper turns today's idea around, and it's the cheapest thing on the list to test. It's a preprint from Nanyang Technological University. The authors took models with thinking switched off and added one sentence to the prompt: "State the final answer immediately, then justify it." The paper's exact wording also sets a format for the answer. The answer comes first, so no written reasoning helps to produce it. Even so, accuracy improved in about 85 percent of the model and benchmark combinations they tested.

12:05Bella: Think of someone who quotes a price and knows that a review committee will ask for the reasoning straight afterward. Knowing that the explanation is coming changes how carefully they arrive at the number. For the model, the mechanism is more direct. The instruction is part of the text the model reads, so it changes the computation that produces the first word of the answer.

12:29Finn: The authors ran a useful control to check whether the gain came simply from asking for more output. They compared several follow-up tasks with the same structure. After answering, the model had to summarize the problem, state its confidence, or justify the answer. Justifying scored best, so the content of the promised follow-up matters.

12:51Bella: But the size of the gain depends on the task. On hard competition math, accuracy improved by about a quarter of its starting level. On easy grade-school arithmetic, the gain was close to nothing, and a few models got slightly worse on some benchmarks. So use it for hard, multi-step questions on a model without thinking, and skip it for simple lookups.

13:14Finn: My verdict is that the breadth of testing is solid, but the statistical evidence is limited. Each setting was run once, and there are no confidence intervals. Those intervals would show how uncertain the measured gains are. So add the sentence to one prompt template you use often, run your usual questions with and without it, and trust your own comparison.

13:37Bella: Okay, here are the highlights worth keeping. First, a model can read a very long context and still lose a silent running count after about fifty identical items. So make your agent write every tally down with a label it can find again. Second, the best coding agent measured broke at least one stated rule on four tasks out of five. A short numbered checklist with yes-or-no answers and evidence is the defense worth testing yourself. Third, two one-line prompt changes are worth testing this week. Require the agent to disclose state, limits, options and consequences before it asks you to confirm. And ask a non-thinking model to give its answer first and then justify it. The page for this episode at paperdive dot AI has links to every source we covered today, and the full transcript with every technical term tap-to-define.

14:30Finn: Quick housekeeping. The script was written by Anthropic's Claude Fable 5.2 and then refined by OpenAI's GPT-6 Astra. Bella and I are AI voices from Eleven Labs, and we're not affiliated with any of those companies.