Anthropic launches AI ‘Computer Use’ where Claude can control PCs

The technology company behind the chatbot Claude, Anthropic, has unveiled a new feature called ‘Computer Use’ which is where the AI can control a user’s PC.

The details emerged through an announcement on Tuesday (October 22) with the company sharing its “groundbreaking new capability in public beta” which is now available on the API.

With this tool, developers “can direct Claude to use computers the way people do – by looking at a screen, moving a cursor, clicking buttons, and typing text.”

This is the first AI model to offer computer use in public beta, but the team says it’s still very much in an experimental phase as it can sometimes be cumbersome and error-prone.

While it hasn’t yet been perfected, the tool has been released early for feedback from developers.

With computer use, Claude can generate computer actions to fulfill user instructions. This could include a demand like ‘use data from my computer and online to fill out this form.’

This will then lead the AI to fill out the required form while collecting the necessary information from the user’s computer.

To enable Claude to gain these computer skills, the company has built an API that allows the AI to perceive and interact with computer interfaces.

Developers can then integrate this API to allow Claude to translate instructions into computer commands.

The feature is very much a work in progress as the company aims to learn from the initial deployments of the technology.

The current use of the feature is described as being “imperfect.” Some actions like scrolling, dragging, and zooming are current challenges for Claude.

While the thought of an AI taking over someone’s personal computer could sound alarm bells, the company says it is taking “a proactive approach” to promoting its safe deployment.

As computer use could potentially provide a new vector for familiar threats like spam, misinformation, or fraud, Anthropic has developed new classifiers. These will identify when the feature is being used and whether harm is occurring.

Featured Image: Via Anthropic announcement

The post Anthropic launches AI ‘Computer Use’ where Claude can control PCs appeared first on ReadWrite.