Working Surfacelink · 17 Nov 2026
05
A newcomer seen from behind stands before a long wall of unlabelled pigeonholes holding one unsigned letter, studying the different hats and coats hung beneath each slot to work out whose it is, the whole scene drawn with wide empty margins on every side.
Concept

Introducing Company Agent Bench

Joshua Qin and Phil Chen · 12 November 2026

Company story · agents · collaboration

Read the original
Takeaway

Concept, a research lab, tested how well four AI agent products understand a company from its chat history, and the best answered 75.9% of 205 questions correctly.

Summary

Joshua Qin and Phil Chen of Concept, an applied research lab, built a test from one month of the lab's own messages in Slack, the workplace chat product. The test asks an AI agent who wrote a message, who was tagged in it, and who wrote each reply. The authors argue that an agent can complete tasks well and still not know its colleagues' roles, priorities and working relationships.

Key points
  • The 205 questions cover about 10 teammates. Each agent searched a copy of the Slack workspace that held only the messages sent before the answer appeared.
  • Four AI agent products were tested at their default settings. ChatGPT Dots scored 75.9%, Grok Bot 72.7%, Claude Tag 67.8% and Devin 66.1%.
  • The products recognised who owns a responsibility more often than what a person cares about. On questions about who wrote a message, they scored 73% to 91% on the first kind and 42% to 54% on the second.
  • Several AI models given only tools to search and read Slack outscored the products. The best, Claude Fable 5.2, scored 82.2%.
Implication

A team that adds an AI agent to its chat tool can expect the agent to know who owns what better than what each person cares about. That advice is an inference and is not the authors' claim. The test covers one small company for one month, and Concept is building its own technology in this field.

Source issue

17 November 2026: as AI agents and staff outside engineering produce the stories, specifications and tools that others build on, a team must name who checks them and who looks after them.