Every week our pipeline scrapes the model catalogs and vendor blogs, then judges each item against one question: would this actually improve something we run in production? Verdicts below. Watch = interesting but unproven claims · Adopt = earned a place · Ignore = noise · Product Input = a feature idea, not a factory change.
30 items judged; 12 survived, 18 dropped. Most intake was generic agentic-pattern commentary (deferred tool discovery, discovery/verification split, lesson-capture, PDF staging, infra checklists) that duplicates practices this Factory already runs — dropped for no novelty. Kept: two genuine security/hardening patterns (Anthropic in/out-of-scope prompt declarations; HF forensic timeline's injection vectors) as Factory Candidates with cheap greppable tests; an MCP tool-name-collision audit as a low-effort Factory Candidate; a CTF-real-framing prompt fix as Product Input gated on an explicit before/after test with its causal claim flagged unverified; skills.sh packs as a Factory Candidate addressing real cross-repo skill-copy friction; claude-vs-claude eval-hierarchy and Vercel AI Gateway cost logs as Product Input, both gated on confirming our own stack before acting. Five Watch items (Shieldstral, Fable 5.2 auto-eval, Anthropic Managed Agents claim, voice-agent eval pattern, Supabase Realtime RLS) are credible but unconfirmed against named products — no action until a concrete gap is verified. No Immediate Factory Upgrade and no incumbent-outage items this week.
Ambiguous or overlapping MCP tool names cause an agent to silently call the wrong tool; adding tier/price disambiguation and auditing name overlap reduces mis-routing.
Why this verdict: Tier 3 (dev.to, unsourced 'nearly half' statistic) but the smallest test is a first-party audit of our own tool set, supplying our own primary evidence in place of the blog's claim — satisfies the verification floor. Directly material given this session's actual tool count. Expected gain Medium, effort Low (one scan pass).
Trajectory-scoring with a strict safety>correctness>efficiency precedence, plus a mandatory real-agent (non-mocked) evaluation layer, changes how pass/fail is computed for agentic QA.
Why this verdict: Tier 3 (dev.to, unsourced methodology) held behind an explicit primary-source verification gate per the floor rule, so it can still register as Product Input. Names a real product and a concrete decision. Expected gain Medium if the current harness lacks safety-first ordering, effort Low to check.
CTF-style verifiable-goal framing, plus explicitly telling the agent the scenario IS real rather than something to verify/check, measurably reduces early-quit/hedging behavior in agentic QA testers.
Why this verdict: Tier 3 vendor engineering blog describing their own internal practice with a specific causal claim, held behind an explicit before/after verification gate per the floor rule rather than adopted outright. Medium-High novelty — concrete prompt-wording fix for a known failure mode (agents hedging on simulated tasks). Expected gain Medium, effort Low (single prompt-line change); clears the ratio only once the before/after test confirms it.
Anthropic Managed Agents reportedly adds per-agent effort controls, a 500-skills-per-session limit, and sub-agent event streaming.
Why this verdict: Tier 1 source, but the item's title/content mismatch and specific numeric claims (500 skills/session) are unverified against our actual stack; credible but not yet tied to a confirmed product dependency.