Both Anthropic and OpenAI dropped a near-identical feature this month. Both call it “Record a skill.” Both claim it turns human demonstrations into reusable automations. Both are selling the same illusion—that watching a screen is the same as understanding a system.
I don't trust the hype. I trust the exploit.
Context: The Agent arms race
Since late 2024, every major AI lab has pivoted toward “agents”—models that not only chat but act: click buttons, fill forms, extract data. The promise is a personal digital worker that learns by watching. Claude’s “Cowork” mode and OpenAI’s Codex IDE agent are the two most visible incarnations. Now they both ship a “record a skill” workflow: you demonstrate a task (screen recording + voice narration), and the model generates a reusable Skill file. Previously, creating a Skill required hand-authoring a SKILL.md—a structured prompt. Now it's point-and-shoot.
The surface story: “Democratizing automation for non-technical users.” The reality: a dangerously oversimplified copy of behavioral cloning that ignores environmental drift, privacy, and execution fidelity.
Core: Systematic teardown
The technical mirage
“Record a skill” is not a fundamental innovation. It is an engineering wrapper around existing capabilities: multimodal perception (vision + audio), intent parsing by a large language model, and code generation. The model observes a sequence of screen states and actions, then compresses that into a policy—a set of conditional rules for when to click, type, or wait. This is textbook behavioral cloning, applied to GUI tasks. The model does not reason about the underlying system; it mimics observed trajectories.
Herein lies the first failure: behavioral cloning fails catastrophically when the environment diverges from the training distribution. If the UI layout changes by a single pixel, if a button is renamed, if the window is resized—the recorded skill breaks. The model has no causal understanding of the task. It only knows that in the training video, after the dialog appeared, a click at coordinates (x,y) happened. When the dialog is slightly different, the model flounders.
Privacy: the elephant in the room
The feature records screen, keyboard, and microphone. That means every password typed (unless masked by the app), every private document scrolled, every confidential conversation spoken—all streamed to Anthropic’s or OpenAI’s servers for processing. The article describing the feature mentions no on-device computation, no data localization, no granular opt-out for sensitive windows. In my experience auditing DeFi protocols, I learned to always stress-test the “trust me” claims. Here, the claim is “we need your full session to create the skill.” The risk: an enterprise HR manager records a new-hire onboarding workflow—complete with payroll data and personal information. That skill, if shared or stored in a cloud repository, becomes a leak vector.
From my 2021 NFT metadata analysis—where I proved 85% of “rare” traits were procedurally generated with a flawed seed—I learned that systems claiming digital value often rely on opaque, fragile backends. The skill recording backend is no different. The user trusts that their data is secured and discarded. There is no evidence it is.
Execution fragility
Even if the task is correctly recorded, the agent's execution environment is inherently non-deterministic. Network latency changes pop-up timing; software updates shift menu hierarchies; antivirus software blocks automated inputs. The model must produce precise mouse coordinates and keystrokes in real time—a challenge that no pre-recorded skill can handle without adaptive reasoning. OpenAI and Anthropic both rely on their largest models (GPT-4o, Claude 3.5 Sonnet) for inference during execution, which adds cost and latency. A typical 30-second recorded task, when replayed, may take 5 minutes due to model deliberation. The “automation” becomes slower than doing it manually.
I've seen this pattern before. In 2020, I simulated Uniswap v2 liquidity pools and showed that the constant product formula hid asymmetric risk for large depositors. The “efficient” model looked good on paper but failed under stress. The same principle applies here: recorded skills appear to work in demo conditions—single app, stable network, no interruptions. In the messy reality of a corporate desktop, they break.
Contrarian: What the bulls got right
To be fair, the feature isn't worthless. Lowering the barrier to creating automations is valuable. For simple, repetitive tasks inside a single, stable application (e.g., “open Outlook, move emails from folder A to folder B, archive them”), recording may succeed 80% of the time. That 80% is enough for part-time power users who would never write a Python script. The potential network effect—a marketplace of shared skills—could accelerate adoption, creating a moat around the platform. Enterprises could standardize SOPs as executable skills, reducing training time.
Furthermore, the fact that both labs converged on the same approach signals widespread agreement that “demonstration-based automation” is the most commercially viable next step for agents. It validates the market.
But validation is not success. The bulls ignore the steep tail risk: one high-profile failure (e.g., a skill that accidentally deletes customer records due to a UI change) could poison trust in the entire category. In the crypto bull market of 2021, everyone believed that algorithmic stablecoins were the future—until Terra/Luna collapsed. The seigniorage model was mathematically impossible to sustain, yet the hype persisted until the exploit. I spent two months reverse-engineering UST in 2022. The lesson: complexity hides fragility. Recorded skills are complex. They will break.
Takeaway: Accountability call
Neither Anthropic nor OpenAI has published any public benchmark on skill execution success rates. They have not disclosed how they handle environmental variance, nor how they audit shared skills for malicious payloads (a user could record a skill that formats the hard drive, then upload it as a “cleanup tool”). The transaction is permanent; the mistake is not. Once a destructive skill runs, the damage is done.
Illusion has a price tag; truth has none. The price of this illusion may be paid in leaked secrets, broken workflows, and eroded trust. I do not need to trust the audit—I trust the exploit. And I am waiting for the first report of a recorded skill that, when replayed, fired off sensitive company data to a third party because the model misrecognized a dialog box.
Until then, treat every “skill” as a hypothesis to be stress-tested, not an asset to be trusted.