MATINTELLECT

👀 OpenAI rolled out a Codex update that changes the very concept of working with an AI agent. The agent now literally sits down at your Windows computer - sees the screen, moves the mouse, clicks through interfaces and types text - while you control it from your smartphone.

How computer use works on Windows:
On May 29 OpenAI added computer use support for Codex on Windows 11 - a first for this platform, on macOS the feature already existed. Codex (built on GPT-5.5) sees the screen through screenshots, understands the state of interfaces and takes actions: clicks buttons, fills in fields, navigates between apps. Tasks you can hand off: testing apps and reproducing bugs through the user's eyes, moving data between programs, automating multi-step workflows without code. Key nuance: on Windows the agent works in the foreground - it takes over the whole desktop. You can't work in parallel - you hand the machine over to the agent. On macOS it runs in the background: things are gentler there

What the agent can do on your computer:
💻 Sees the screen in real time - analyzes screenshots, understands what's open and in what state, builds the task context
💻 Controls the mouse and keyboard - clicks buttons, drags elements, fills in forms, types text
💻 Works across apps - moves data, switches between programs, tests and reproduces bugs in a specific environment

How to control it from your phone:
⚡️ Connect via QR code - Codex Desktop generates a code, you scan it in ChatGPT on iOS or Android - your phone becomes a remote
⚡️ Launch tasks remotely - you type what needs doing from your phone, the agent starts working on the PC, you go about your day
⚡️ Talk to the agent along the way - it asks clarifying questions, sends updates, you approve the next steps - you're not tied to your desk

You can run several parallel threads on one PC and monitor them all from your phone. Security: the agent runs inside Windows Sandbox with limited access to files and the network. What the agent CAN'T do: perform actions that need admin rights, confirm system security prompts, work with terminal apps. At launch it's not available in the EEA, the UK and Switzerland

💼 First launch: Codex Desktop → connection menu → QR code → scan in ChatGPT with the same account → authorization

⭐️ Satya Nadella, CEO of Microsoft:

"We're moving from the era of copilots to the era of agents. Copilots work for you - agents work with you"

🙃 Now the setup literally looks like this: me on the couch with my phone, the agent at the computer doing the work. This used to be a metaphor from articles about "the future of AI". Now it's a QR code, ChatGPT and a Codex window on the desktop. The main question is no longer "can the agent control a computer" - it's "what am I going to do while it works"

Instagram | YouTube | Threads

Share:

No comments yet

Leave a Comment

Fields marked with an asterisk (*) are required