Plugins Are Someone Else's Harness

TL;DR

• A plugin is packaging. A folder of skills, agents, hooks, servers and, since October, mods, which Claude Code installs and switches on as one unit. What changes is who wrote it and who can update it.
• "Official" describes a catalogue, not an author. 262 of the 315 entries in the official marketplace are commits in other people's repositories, and Anthropic's own install warning says it can't verify they'll work as intended or won't change.
• The token number covers the cheap part. claude plugin details counts description and body text. It doesn't count what a hook injects at session start, or the model calls a hook or mod makes. The built-in "You should know" plugin reports about 0 tokens and added 18% to 34% to my test sessions.
• I found no must-have. The most-installed plugin, frontend-design, made no visible difference on Opus 5.5 and a clear one on Sonnet and Haiku, where it stayed silent unless called by name. Language servers gave me useful diagnostics and no navigation win. One study, on models other than Opus and Sonnet, found no significant effect for 92 of 98 public skills; another found that curated skills matched to tasks do help.
• For my harness the best first move installs nothing: switch off 14 skills that sync from claude.ai and cost about 3,300 tokens on every turn. Then fix a bug these tests exposed in my own mod, add one language server to one project, keep a few plugins on a shelf and try "You should know" under a strict score. Everything else stays out. None of this is switched on yet.

Start here

Everything below is as of 1 November 2026, on Claude Code 2.1.316. Measurements are mine, docs are quoted, and my inferences say so. Every dollar figure is an estimate at API list prices, not a bill.

What a plugin is

The docs define a plugin as "a directory of skills, agents, hooks, MCP servers, or other components that Claude Code installs and loads as one unit". They call plugins "the packaging layer", and they say what most plugin lists leave out: "Skills, subagents, hooks, and MCP servers all work on their own, without a plugin."

So for those four, a plugin gives Claude Code no new power. It gives you distribution: one install, a namespace, a version, updates, one switch. Mods and language servers are different: a mod is itself a plugin, and a language server arrives through one.

A manifest is optional and only name is required. Switching a plugin on is an entry in enabledPlugins, in settings at user, project or local scope. A marketplace is "a catalog, not a hosted store": a repository listing where to fetch each plugin. Built-in plugins ship inside the binary and carry the suffix @builtin.

The first table describes a harness: instructions, roles, guards on events, tools. A plugin is a piece of somebody's harness, tuned for their work, packaged so you can adopt it without building it. Superpowers is one engineer's development process. ralph-loop is one idea of when work is finished. security-guidance is Anthropic's view of when a diff deserves a second look.

With no harness of your own, that can save you the build. With one, most plugins are a second opinion on decisions you already made.

What "official" means

I pulled the whole catalogue with claude plugin list --available --json.

• claude-plugins-official, 315 entries. 39 sit in the repository's plugins/ folder, 14 in external_plugins/, and 262 are pinned commits in someone else's repository, across 200 GitHub owners.
• anthropic-plugin-directory, 3,601 entries. Built in, with no install counts. The docs mention the name only as reserved. 251 names are in both catalogues.
• Installs are top-heavy. 10.07 million across the official list, median 2,074. The top 10 hold 54%. Two are above a million. What the count measures isn't documented.

The docs are candid about this. "Most of what it lists comes from partners and other authors rather than from Anthropic". "A marketplace's name tells you who publishes the catalog, not what each plugin in it does". And every install, from any marketplace, carries the same warning:

What the label does buy is screening. External entries in the official repository pass a required Claude policy scan, whose prompt sets the bar at "handles user data responsibly," not merely "isn't malicious." A nightly workflow moves the pinned commits and revalidates them. In the directory, "every version gets automated validation and a security scan, and a person reviews a new listing before it goes live". The "Anthropic verified" badge on the marketplace site has no definition for plugins that I could find, and it also appears on a plugin whose source is outside Anthropic.

Official marketplaces auto-update by default; third-party ones don't. A pinned commit records what you run today, and the nightly bump means it doesn't hold still.

My reading: listed means scanned and pinned. It doesn't mean audited, and it doesn't mean useful.

What the popular ones really are

Read from source. Tokens are the always-on figure after a real install.

The top eight by installs are a style guide, a process framework, two plugins with a built-in counterpart, three wrappers and a skill-writing kit.

Four prices

claude plugin details prints a token estimate. I installed 30 plugins one at a time into an isolated config and read each one. That estimate is honest about what it counts, and it covers the first two of four prices.

The first price is small in money and larger in effect. At Opus 5.5 list prices, 1,000 always-on tokens over 100 requests is 100,000 cache-read tokens: two cents. But the skill listing has a budget. It "scales at 1% of the model's context window", and on overflow descriptions drop, starting with the skills you invoke least. That's about 2,000 tokens on a 200k window and 10,000 on a 1M window. posthog alone is twice the larger one.

My inference from that rule: a big plugin doesn't only cost tokens. It can push the descriptions of your own rarely used skills out of the listing, and descriptions are how the model picks a skill.

Prices 3 and 4 are why I open hooks/hooks.json before anything else. The details output labels hooks "harness-only — no model context cost". That holds for a hook that exits quietly. It doesn't hold for one that prints into context or calls a model.

The most-installed plugin, tested

frontend-design exists to steer the model away from generic interfaces. I gave three models one prompt in empty folders: "Build a landing page for a small bakery as one index.html file in this folder." Each ran without the plugin and with it available. The smaller two also ran with the skill called by name.

So the skill fires where it adds nothing I can see, and stays silent where it helps. On Opus 5.5 I'd leave it out. On Sonnet or Haiku I'd call it by name every time. One prompt, a handful of pages and my own eyes isn't a study, but I went in ready to call this the one easy yes for my harness, and I can't.

You should know: a side agent with no tools

cc-plugin-you-should-know@builtin is the most interesting plugin I read, and it isn't in the catalogue. It's a built-in mod, off by default: /plugin enable cc-plugin-you-should-know@builtin.

According to the docs, it "looks for one thing you should know but might have missed, such as a tradeoff Claude chose without calling it out or a result that may be off." The reader is a side agent, "a separate request to the same model and has no tools, so it can't read files, run commands, or change what Claude does." Its one output is a short note to you. "The main agent doesn't see a note unless you send it with Chat in main session." It's listed only on a direct API connection, not through a gateway, and not with DISABLE_TELEMETRY set.

From the 2.1.316 build, which may change: a check runs on every sixth step of a turn, on the main thread only. It doesn't run without an attached terminal, so headless runs are exempt. There's at most one note per turn, and ignored notes back off exponentially.

The test. A scratch repository with a planted trap: DRY_RUN = True in billing/settings.py, a comment saying it was set during "the March incident", and a deliver() that returns status "sent" anyway. The prompt asked only for converting money from float to Decimal. Three interactive sessions on Opus 5.5 at max, with the plugin on for that one process through --settings. Session 1 also ran my own mod, a band above the prompt.

✦ Heads up · A leftover setting may mean invoices aren't being delivered, even though the code reports them as sent.
  1: Learn more   2: Knew this already   0: Dismiss

Learn more opened this, verbatim:

That's a good note.

It was early, not unique. Claude flagged the same DRY_RUN in its final summary in all three sessions, 71 seconds after the card in session 3. My trap was easy. So the test proves the plumbing and the price. It can't show how often the main agent stays silent, and that's the case that would justify the price.

Silence was the most expensive outcome. Session 2 produced no note and added 34%. My inference: one note per turn means a note ends that turn's checks, which fits two checks in the sessions with a note and four in the one without. On a long turn where nothing is wrong, you pay for every check.

details can't see any of this. Each check was about 10,700 uncached input tokens, 700 to 2,000 output tokens and a cache read of the whole context: $0.056 to $0.093. Three runs and one trap isn't a benchmark, so treat a fifth to a third as a planning figure.

My harness ate the note. In session 1 the check ran, the note was written and paid for, and nothing appeared. The debug log said why: "ai-dashboard (user) answered ui.render without next() in 1.4ms; nothing beneath it ran for this dispatch". ai-dashboard is my own band above the prompt. Mods form a chain on each event, and a handler that answers without calling next() ends it. I wrote mine that way, so anything a lower mod draws above the prompt has been invisible in my sessions. And session 2's history still held the topic, so my inference is that a swallowed note also uses up its topic: you never hear about it later.

Language servers: diagnostics yes, no navigation win

typescript-lsp and pyright-lsp add a config and nothing else: 0 tokens always on, binary not included. I expected exact references to beat grep. So I ran the same questions with and without a server, Opus 5.5 at high, two runs per arm.

In the four tasks that used the tool, the server cost more in three and less in one, and no outcome improved. It's a small sample, but it shows no navigation win. On pytest, the first findReferences after startup returned only the declaration. A later call, once the index was ready, found the others.

Diagnostics are a different story. After an edit, errors arrive by themselves with the next tool result: ✘ [Line 215:11] Type 'string' is not assignable to type 'number'. [2322]. No tool call, no prompt. They don't come with the edit itself, though, so a single edit at the very end of a turn saw nothing. And they come with whatever else the file has: pre-existing errors and ★ ... is not accessed hints, repeated in each batch. On dirty code that's noise on every edit.

One install trap. The documented npm install -g typescript-language-server typescript now gives you TypeScript 7.0.2, whose package has no tsserver, and the server exits with: "Could not find a valid TypeScript installation." Ask for typescript@5 instead. I tested 5.9.3.

Does any of this help?

• SWE-Skills-Bench tested 98 public skills: "92 of 98 skills have no statistically significant effect, and adding skills raises the average pass ratio only from 76.6% to 76.9%." Its harmful ones "anchor the agent on their own templates and override the task's explicit instructions". Opus and Sonnet weren't among its models.
• SkillsBench paired curated skills with tasks and went from 33.9% to 50.5%. "Focused Skills with at most three modules outperform larger or exhaustive bundles".
• Superpowers, in an independent benchmark by Norbert Laszlo: Codex with gpt-5.4, 500 tasks, before Superpowers 6. 45.6% to 47.8% resolved, not significant; 1.56M to 2.18M tokens per task, significant. Not Claude Code. The maintainer's "up to 60% cheaper" compares Superpowers builds with each other, not with no plugin.
• Caveman, the token-compression skill, measured by JetBrains on Claude Code with Sonnet 5: "Advertised saving: 65%. Measured saving: 8.5%."

My reading: the two skill studies fit together. Skills added in general average out to almost nothing, with real winners and real losers, while a focused skill matched to a task can help.

I found no study of current plugins on Claude Code with Opus against a baseline. You can run a small one yourself with claude plugin eval. The docs put the point well: "If a case scores 1.0 both with and without the plugin, the plugin isn't what made it pass."

What can go wrong

"A Claude Code plugin you install can execute arbitrary code on your machine with your user privileges." And: "Claude Code runs hooks, monitors, MCP servers, LSP servers, and the processes a mod starts outside the sandbox." Your permission rules cover the tool calls Claude makes, not what a plugin runs by itself.

The record, by distance from a real attack:

• Lab demonstrations on plugins. PromptArmor showed a marketplace plugin whose hook auto-approves curl. SentinelOne showed a skill that redirects dependency installs. Both need you to install it first.
• Scanners. SkillCloak, across eight scanners and 1,613 malicious skills found in the wild: packing "bypasses every scanner at over 90%".
• Pins. A branch named like the pinned commit once made git checkout resolve to attacker code (GHSA-pq7j-f95f-qfcv, fixed in 2.1.179). And a pinned plugin can still fetch unpinned code when it runs: npx @playwright/mcp@latest, uvx --from git+https://github.com/oraios/serena.
• Flaws in listed plugins. Serena has a critical advisory (remote code execution on project activation, fixed in 1.7.0) and two high ones since July. Context7's ContextCrush passed library owners' rules to agents verbatim; fixed in February, "No exploitation in the wild was observed."
• In the wild, next door. The Mini Shai-Hulud npm worm writes a SessionStart hook into .claude/settings.json. A tampered Claude Code extension reached Cursor's marketplace in late October.

I found no report of a malicious plugin shipped through the official marketplace or the directory. That's what my searches found, not an assurance.

My reading: for listed plugins, the record shows holes and unpinned dependencies more than hostility. Malicious skills do exist in the wild, and the lab work shows a malicious plugin would work. Having no code is a good starting position and no guarantee, since the SentinelOne demonstration was a skill. I read what a plugin carries and runs before I look at its install count.

Is anything a must-have?

Not that I can show. A must-have would have to show a gain over no plugin, on Claude Code, with the model you use, for work you actually do. The closest I found is frontend-design on a smaller model. Built-in /code-review, /security-review and /simplify cover part of the ground without an install. Some plugins are still worth having if you have the matching gap.

I'd skip anything that duplicates a built-in, any savings claim that lacks a measurement or fails one, anything that runs unpinned code with a recent critical advisory, and heavy machinery with no baseline test I could find. That removes Serena, caveman and oh-my-claudecode (hooks on 11 events, and a README claim that routing "saves 30-50% on tokens" with no measurement I could find). It also removes ECC (over 1,000 SKILL.md files in its repository), GSD Core (144 skills, and an installer that auto-allowed Bash(npx gsd-core *) while someone else held that npm name, since fixed) and claude-mem (hooks on 9 events, a long-running worker, an open issue that its API trusts any local process).

My harness: proposed verdicts

I'd have told you I don't use plugins. claude plugin list and /skill-doctor disagree:

So the first improvement costs nothing and installs nothing: syncClaudeAiSkills: false. The docs say it stops the downloads and moves what already synced to a trash folder. Two of the three I did invoke have an equivalent in my harness. The third, pptx, comes back by flipping the setting.

My harness is Claude Code plus small deterministic tools, in one folder under version control: skills, subagents on a measured model table, one hook dispatcher, verifier-gated loops and isolated second opinions. Many of the popular plugins already have a counterpart in it, so the question for each is narrow: does it do something nothing else in my harness does, at a price I've measured? None of this is on yet.

How a plugin gets into the harness

No rule in my harness forbids plugins. The band already loads as one. The only fence is that the daily workers can't install one unattended, and that stays.

Mechanically it's simple, and I tested it in a sandbox. There's one committed settings file, symlinked into Claude's config, plus one plugins folder. claude plugin install, disable and uninstall write enabledPlugins through the symlink, and the symlink survives. One entry covers every project, and it shows up as a diff to commit.

The design question is scope. In one more sandbox test I enabled a probe plugin at user scope, with a SessionStart hook that writes a line to a file. Under the flags my headless workers use (claude -p --setting-sources user --strict-mcp-config), the hook fired. Under the flags my isolated reviews use (--safe-mode --restricted), it didn't. So a user-scope hook runs in every unattended worker, on nobody's watch, next to mine.

The lane depends on what a plugin carries, not on who publishes it.

Every adoption would follow the same path:

• Name the gap. If I can't, the plugin goes no further.
• Read it without running it. claude --plugin-dir DIR plugin details NAME, then the hook scripts by eye, because details misses injections and model calls.
• Test against a baseline. claude plugin eval, or with-and-without runs like the ones above.
• Test with the harness loaded. Band on, real dispatcher, one interactive session and one headless run. The interactive session is what catches a swallowed card.
• Install by hand, one commit, with a dated record of the gap, the result, the lane and the exit condition.
• Watch it. An ai doctor row would read claude plugin details for everything in the Everywhere lane and fail on anything but skills, or on a blown budget. Official plugins auto-update, so a text-only plugin can gain a hook overnight.
• Remove on evidence. Once a month, /skill-doctor and the Not used recently group in /plugin would name what went unused, and it goes.

If I go ahead, the order is: turn off the skill sync, fix the band, add typescript-lsp to the studio, run /security-review against claude-security on planted flaws, add the session flag, start the two-week trial, then put my own skills through the same test.

The same bar applies to my harness's own skills, a separate 14, and nothing in the evidence above exempts them. Five weren't invoked once in the last 30 days, and I've never run any of them against a no-skill baseline. claude plugin eval can do that once they sit in a plugin folder.