cotomi Act: Learning to Automate Work by Watching You
For users of browser automation, this work demonstrates that passively learned behavioral knowledge can enhance task execution, but the approach is incremental as it builds on existing agent scaffolds and knowledge management techniques.
cotomi Act is a browser agent that achieves 80.4% task success on WebArena (exceeding 78.2% human baseline) by combining reliable execution with organizational knowledge learned from observing user behavior, which improves task performance as knowledge accumulates.
What if a browser agent could learn your work simply by watching you do it? We present cotomi Act, a browser-based computer-using agent that combines reliable multi-step task execution with persistent organizational knowledge learned from user behavior. For execution, an agent scaffold with adaptive lazy observation, verbal-diff-based history compression, coarse-grained actions, and test-time scaling via best-of-N action selection achieves 80.4% on the 179-task WebArena human-evaluation subset, exceeding the reported 78.2% human baseline. For organizational knowledge, a behavior-to-knowledge pipeline passively observes the user's browsing and progressively abstracts it into artifacts (task boards, wiki) exposed through a shared workspace editable by both user and agent. A controlled proxy evaluation confirms that task success improves as behavior-derived knowledge accumulates. In our live demonstration, attendees interact with the system in a real browser, issuing tasks and observing end-to-end autonomous execution and shared knowledge management.