Things AI Is Surprisingly Bad At

An AI system can produce a useful explanation and still make an elementary error in the next sentence. Fluent output does not tell you which parts were checked. The practical response is to identify the failure modes relevant to the task and give the workflow an appropriate way to detect them.

These are recurring reliability concerns, not permanent impossibility claims about every model. Capabilities depend on the model, prompt, tools and surrounding application. A system with a clock, browser, calculator or executable code can behave differently from an isolated model answering from its trained parameters.

Current facts need current evidence

A model’s training does not provide a continuously updated view of the world. A date supplied in the session, a clock tool and a live search each provide different evidence. Even when a system knows today’s date, that does not establish that its account of a product or recent event is current.

For time-sensitive work, identify the relevant source, retrieve it when necessary and check its date and scope. Ask whether a statement describes the current release, a historical version or a prediction. Keep that distinction visible in the final answer.

Exact text operations deserve exact checks

Letter counting, string matching and precise formatting can expose errors that seem surprising beside a strong prose answer. Tokenization is relevant to how a model represents text, but it does not establish that a model cannot count characters or that every error has the same cause.

If an exact count matters, use a deterministic operation and inspect what it counted. Characters, bytes, words and tokens are different units. Define how punctuation, Unicode and repeated matches should be handled before treating a number as authoritative.

For a generated artifact, verify the artifact itself. A model’s statement that it produced a certain number of rows is not a substitute for counting them. The same principle applies to required fields, valid JSON and unique identifiers.

Arithmetic and derivations need suitable tools

Models can solve many mathematical problems, but a plausible derivation can contain an arithmetic or algebraic error. Difficulty and reliability vary. A simple-looking calculation is not automatically safe to accept without checking.

For consequential arithmetic, use an appropriate calculator, spreadsheet or program. Then validate the input values, units and formula. A correct computation over the wrong assumptions can still produce a wrong answer.

When a result is surprising, check it with an independent method or a known bound. Keep the original data and the calculation reproducible. A tool result helps establish what was computed; it does not decide whether the computation answers the actual question.

Confidence is not a measurement of correctness

A model can express uncertainty or confidence, but its phrasing alone is not a calibrated probability. It can also accept a false premise and elaborate on it. Asking for a confidence score does not by itself solve either problem.

Ask which evidence supports the claim, what remains unknown and what observation would change the conclusion. Require citations when the task depends on external facts, and inspect whether those sources support the specific statement.

An independent review can catch errors, especially when it checks evidence and assumptions rather than merely restating the answer. Agreement between models is still not proof: they may rely on the same source or make similar mistakes.

Spatial reasoning and visual inspection have limits

A multimodal model may describe a diagram accurately while misreading a small label, relative position or count. An image may omit the geometry or scale needed for the question. Performance on one kind of visual task does not establish reliability on another.

Use the original-resolution artifact when details matter. For interfaces, inspect the rendered result at relevant sizes and states. For engineering geometry or measurements, use the underlying structured data and established checking tools where available.

Distinguish a visual impression from a verified property. A screenshot can help assess appearance, but it does not alone establish keyboard behavior, data correctness or every interaction state.

Long outputs can drift from the task

A response may start correctly and later introduce an unsupported assumption, omit a constraint or claim completion prematurely. More text does not automatically mean more complete reasoning.

Keep acceptance criteria explicit, divide work when that makes intermediate results easier to inspect and verify the final deliverable against the original requirements. Reopen authoritative sources when exact wording matters rather than relying on a compressed recollection.

Design the workflow around the error that matters

Choose checks according to the consequence and likelihood of failure. Exact text operations need exact checks. Current facts need current sources. Numerical claims need reproducible calculations. Software changes need tests and review that reflect the intended behavior.

The aim is not to dismiss AI because it can make small mistakes or to excuse those mistakes because it can do difficult work. It is to use its capabilities while making important errors observable before the output is relied on.