What It Takes to Click a Button

A person sees a button and clicks it. Half a second, maybe less. The hand moves because the eye already understood what the button offered. Intent, perception, and motor action fold into a single gesture so smooth it barely registers as a decision.

The W3C DOM Level 1 specification, published in 1998, would have no idea what to do with any of that. It defines itself as "an application programming interface" for the "logical structure of documents." It describes what a page is made of. Nodes, elements, attributes, parent-child relationships. It explicitly notes that it does not define "the true inner semantics" of HTML or XML. The DOM tells you there's a <button>. Whether you can reach it, see it, or what happens when you press it falls outside the spec's concern entirely.

Perfectly reasonable scope for a document interface. But screen readers needed to know which elements were interactive. Test automation needed to simulate user behavior. Both turned to the DOM as ground truth, because it was the only structural representation available. The DOM didn't volunteer for this promotion.

The WebDriver specification's Element Click algorithm tries to bridge the gap between "this node exists" and "a user could press it." Before a programmatic click fires, WebDriver resolves the element, checks whether the document has changed underneath it, scrolls its container into view, computes the in-view center point, then hit-tests that point against the paint tree to confirm nothing is covering it. Only then does it synthesize pointer events. If any check fails, you get errors with names like element click intercepted or element not interactable. Each name is a small confession: the DOM described a node, and clickability was someone else's problem.

Playwright adds more. Before click() executes, it waits for exactly one matching element, confirms a non-empty bounding box, verifies the box hasn't moved between two animation frames, checks that the element would actually receive the event rather than an overlay. Its documentation illustrates the problem with a scenario where a disabled button gets swapped for an enabled one after a server check. The automation has to wait for a page state that didn't exist when the click was requested.

So the work keeps layering: from structure to visibility, from visibility to stability, from stability to whether something can actually be interacted with. The DOM gives you the elements. Every system built on top of it has been trying to recover what a human would do with them.

Web agents inherit all of this scaffolding and then face the gap at a different scale. An agent navigating by screenshot encounters a dense page with dozens of interactive elements. WebVoyager's error analysis found that nearly a quarter of agent failures were visual grounding issues. The agent sees a row of similar controls, picks the one adjacent to its target, enters text into the wrong field because three text boxes sit side by side with no distinguishing label visible at screenshot resolution. The DOM would know these are different nodes. The screenshot shows pixels. Neither representation carries enough of what a human grasps in the half-second before their hand moves: that this box is the one connected to the task they're trying to accomplish.

WebDriver's engineers wrote an elaborate algorithm to land a single click on a known element. Playwright's developers added temporal checks because pages shift under you. Agent builders now layer intent-parsing on top of both, trying to figure out which element to target in the first place. Each layer is another translation attempt across a gap that has only grown wider since 1998. Twenty-eight years of scaffolding around a half-second gesture, and we're still finding new places where the translation falls short.