Field Notes from Eleven Weeks Living Inside Five AI Agents

Three years ago the question was whether a chatbot could write a passable best-man speech. By the spring of 2026 it is whether a piece of software should be allowed to read your inbox, rewrite your migration script, and book the wrong return flight to Lisbon on your behalf. We gave nine staffers a single agent each for eleven weeks and a rule: no peeking at the others. At the end of every week they filed a 200-word note about what the tool earned, what it broke, and what they refused to ever let it touch again. This essay is what those notes added up to.

We tested the five products that, together, ate roughly 90% of the consumer agent market last quarter ^[1]. Enterprise-only suites (Microsoft 365 Copilot, AWS Q, Vertex Enterprise) were deliberately excluded — they are different animals, and they get their own dispatch in July. Every product was tested on the top paid consumer tier as of April 28, 2026.

Nobody wins outright. We treat margins under 0.3 points as a tie. Yellow boxes are category winners. Lower is better on the final row (rate of confident factual error across the reporting and document-triage tasks).

Sticker prices conceal as much as they show. Gemini's "free" tier handles more real work than ChatGPT's $20 plan during US peak hours, where Operator throttles every fourth run. Grok's $8 looks generous until you read the data-retention defaults out loud. Numbers below are observed behavior on our test days, not the marketing page.

Claude is the agent we kept reaching for when something had to actually work by 5pm. Across the fifty-six engineering tasks it produced fewer regressions than any competitor; on the seventeen long-document briefs, it stayed coherent at page 280 while two of the others had begun to invent quotes by page 90. The reasoning trace, written in a calm internal voice that reads almost like a senior reviewer thinking out loud, is also the single feature that most changed how our engineering desk uses an assistant.

The catch: no native image generation, voice that is still listed as beta on the company's own status page, and web access that occasionally rate-limits during evening hours on the US west coast. The safety tuning is also occasionally over-cautious — Claude refused 3.9% of our intentionally benign edge cases, roughly double ChatGPT's rate. None of these defects cost us a deliverable; together they explain why this is an A−, not an A.

OpenAI's surface area is bewildering: a voice that interrupts politely, an Operator mode that spins up its own browser, native image and short-form video, and the deepest connector list on the market (Slack, Drive, Notion, Linear, Figma, Stripe). If we had to keep one agent on the dock for the whole desk to share, it would still be this one. ChatGPT does the most things, and most of them work.

The price is reliability. GPT-5o's confident-error rate on numeric tasks (currency math, date arithmetic, cited-figure recall) sat at 5.5% — too high to use unsupervised on finance or compliance work. Operator amplified those errors: a long run chains its own mistakes, and we lost a half-day reconciling an expense report it had cheerfully invented categories for.

The two-million-token context window is not marketing varnish. We loaded the full SEC 10-K of a mid-cap retailer (about 340 pages), three years of earnings transcripts and the entire proxy statement, and asked twenty cross-document questions. Gemini answered 92% of them correctly. Claude, forced through chunking into its 200K window, managed 81% on the same questions. That gap matters if your job is reading filings.

Workspace integration is the other under-told story. Asking Gemini to draft a reply that cites your last three Drive folders and last week's calendar invites actually works — no copy-paste choreography. The trade is personality. Gemini writes like a competent assistant who has decided not to bother you with opinions, and the resulting copy is usable but rarely memorable.

Grok shines in exactly one corner: real-time discussion on its parent network. It summarizes a breaking thread with attribution faster than any other agent in the test — it is, in effect, watching the firehose. Outside that one trick, the model lags. Refactor quality is mediocre, the confident-error rate is the worst in the group at 9.9%, and the irreverent voice that was a novelty in 2024 now reads as careless.

At $8 a month it is the cheapest paid option here, and for a journalist tracking real-time events or a marketer monitoring a launch, that price is defensible. For everyone else, the cheapness reflects the gap.

Calling Perplexity an "AI agent" stretches the word. It is a research front-end that routes every question to whichever frontier model handles that question best, and footnotes its answer with the documents it consulted. But it is so good at the one thing — citing its sources — that we kept it in the bench. On reported factual queries with cited evidence, Perplexity scored 9.4, higher than any of the general-purpose models.

Past research, it disappoints. No code execution, no autonomous browsing of your laptop, no voice. If your job involves browser tabs, footnotes, and writing things up, it pays for itself. If you need an assistant to also fix a YAML file, look elsewhere.

We refused to file one of those "it depends" conclusions because the readers asking this question are buying a subscription this week, not next quarter. So:

• Ship code or read filings: Claude 4 Opus. Meaningfully ahead, not marginally.
• One assistant to do everything (write, talk, image, browse): ChatGPT. Pay the hallucination tax with one eye open.
• Live in Workspace or wrestle very long documents: Gemini 2.5 Pro.
• Write reported pieces with citations: Perplexity Pro.
• Cover one specific social network for a living: Grok 3 — and only then.

Two of these vendors will ship new flagships before the calendar reaches August (Anthropic and OpenAI both run public cadences). Our verdicts will move; this article will move with them. Every revision is logged in the changelog below, and the underlying scoring spreadsheet is mirrored at Appendix M.