Know the Code, Write the Spec

The study

What predicted success

• CS knowledge and writing are separate skills. They barely correlate with each other (r = 0.13).
• With both in one model, CS carries about twice the unique weight: 12.5% of the variance on top of writing, against 5.9% for writing on top of CS.
• Writing also overlaps with general reasoning (r = 0.37). With reasoning controlled, its link drops below significance.
• About half of writing’s link to the score (52%) ran through prompt quality. Better writers wrote better-rated prompts, and better-rated prompts went with better apps.
• On the meal-planner task, which needed loops, totals and thresholds, CS counted more (r = 0.32) than writing (r = 0.20).
• The CS test was pseudocode: tracing, definitions, completing code. Concepts, not the syntax of any one language.
• Together, CS and writing explain about 21% of the variance in vibe-coding scores. The rest is something else.

Theo Weidmann, one of the authors, to ETH News: “Good computer scientists can plan an app’s structure more precisely and debug potential errors faster.” And: “People who formulate clear and structured prompts achieve better results, while unclear or imprecise wording is more likely to lead to defective software.”

Frequent LLM users scored lower

Frequent users were not weaker at CS or reasoning. The gap shows up in writing and in vibe coding.

The authors’ three possible explanations:

1. LLMs weaken students’ ability to express themselves.
2. Students who write less well use LLMs more.
3. Both.

Not established: the direction. The data are correlations from one sample at one point in time, with LLM use self-reported on a five-point scale.

Other evidence

The common thread:

• Judging output takes knowledge. Agents change code that was already correct, and beginners misjudge generated code.
• Content beats wording. What a prompt specifies matters more than how technical it sounds.
• Handing over the thinking has a cost: less learning, and a poor read on your own speed.

What to do with it

How I apply it

• The spec and the architecture come first. Agents build from them.
• One model writes, other models review it independently, and the decision stays mine.
• When all three assistants hit their limits at once, those days go to work without them.

What is not known

• Students, 15-minute tasks, a lab. Professionals on real codebases were not studied.
• The code was hidden. Workflows where people read and edit it may give different results, as the authors note.
• One model, Claude Sonnet 4. Newer models were not tested.
• Several participants ran out of time close to a working app.
• The writing test is new. Rater agreement was 0.735, under the preregistered 0.75 target.
• Everything in the ETH study is correlational, and LLM use was self-reported.

Sources

• ETH News: What skills do people need to successfully program with AI? 30 April 2026.
• Thorgeirsson, Weidmann, Su: Computer Science Achievement and Writing Skills Predict Vibe Coding Proficiency. CHI ’26. DOI 10.1145/3772318.3791666.
• ETH SRI Lab: coding agents on already-fixed issues. 23 March 2026.
• Anthropic: How AI assistance impacts the formation of coding skills. 29 January 2026.
• METR: Measuring the Impact of Early-2025 AI on Experienced Open-Source Developer Productivity. July 2025.
• METR: update on the developer productivity study. 24 February 2026.
• Zi et al.: “I Would Have Written My Code Differently”: Beginners Struggle to Understand LLM-Generated Code. FSE 2025.
• Lucchetti et al.: Substance Beats Style: Why Beginning Students Fail to Code with LLMs. NAACL 2025.