Buttons are pictures of commands.
The graphical user interface replaced typed commands with visual metaphors: windows, icons, menus and a pointer, pioneered at Xerox PARC and carried to the mass market by Apple’s Macintosh in 1984. Its achievement was real and civilisational in scale — it let anyone use a computer without learning the machine’s grammar, which is exactly why computing escaped the research lab. The price was hidden inside the bargain: every action routed through a picture means every action is designed for eyes and hands, not for anything else that might want to act.
That bargain is now being renegotiated. An agent cannot see a button the way you do; software built to drive GUIs has to scrape pixels or accessibility trees, guess at what is clickable, and break whenever a designer moves something two points to the left. Text is simply not where a GUI lives. This is why the current generation of agents prefers the terminal and generated command line interfaces, and why projects now take the opposite route to history: when software only offers a GUI, generate it an API-grade CLI so a model can drive it. The pictures were always the front; the commands were always underneath.
Source: Myers, B.A. (1998), “A Brief History of Human-Computer Interaction Technology,” ACM interactions, 5(2).