Google: the AI agent is now your problem

Google’s terms now say that an AI agent can act against its instructions. Google’s contract makes the company that runs the agent responsible for every action the agent takes. So an agent is only as good as the people who run it and know where it fails.

The admission comes from new guidelines for AI agents, which Google announced to its Workspace customers alongside updated terms.¹

Google describes agents as a new technology that may carry significant risks. An agent may take actions that go against its instructions or that its user did not expect.

Agents are also more vulnerable to outside attacks, such as instructions planted in content to take over the agent. Google even bars customers from letting agents conceal that they are AI. An agent may not make more copies of itself than its task requires without the customer’s explicit permission.¹ Rules like these are written for software that acts on its own.

Google strongly recommends that a person approves any agent action with serious legal, financial or physical consequences for people.¹

Said in public, written into the terms

AI vendors’ public message promises agents that get things done. Here is that message set against the vendors’ own terms.

• Google presents its agent as one that “plans the work” and “brings back something finished”.² Google’s terms make the customer solely responsible for every action an agent takes.³
• OpenAI offers companies “AI coworkers”.⁴ OpenAI’s terms make the customer responsible for checking whether the output it receives is accurate.⁵
• Anthropic’s chief executive says models could soon do most of a software engineer’s work.⁶ Yet Anthropic supplies its services and their output “as is”, with no warranty that the output is accurate.⁷

The public statements talk about agents in general terms. They stay the same whatever product a company buys and however it uses it. They shape what the public, investors and companies expect AI to deliver.

The terms go into detail. The customer decides whether an agent is right for the job, what data and systems it may access and how it is supervised once it is running.³ If an agent breaks Google’s usage rules and a third party takes Google to court over it, the customer pays Google’s costs.⁸ A serious breach can cost the customer the whole service, because Google may terminate the agreement immediately if it considers that a breach cannot be remedied.⁹ Google also reminds customers that they pay for whatever their agents spend.¹

In one respect, Google’s terms do protect the customer.

For paying customers, Google covers unmodified text its AI writes against claims that it infringes someone else’s intellectual property, with some exceptions.

Responsibility for what agents do rests with the customer alone.³

The protection stops exactly where the promise starts.

So which one counts?

A public statement presents what the technology can do. A contract sets out what the vendor answers for. Google’s and OpenAI’s contracts state that only what is written in them counts.^{8 10}

European law limits some of what the contracts leave to customers. Under EU rules on misleading business-to-business advertising, courts can require a vendor to prove its factual claims.¹¹ Under the EU’s new product liability rules, makers of software will be liable to people harmed by a defect, whatever the contract says.¹² Polish law, for example, voids any clause that excludes liability for damage caused intentionally.¹³

Even so, the risk of an agent’s actions stays largely with the company that runs it.

Checking stays with people

The company running the agent also has to check its results, and independent tests show how much depends on that check.

On banking, consulting and legal tasks, the top agent in one test succeeded at the first attempt about a quarter of the time.¹⁴

A model can also return different results for the same input.¹⁵

Someone has to spot a wrong result before it causes damage. Spotting it takes a person who knows what the agent does well and where it fails.

The guidelines name the steps where Google recommends getting that person’s approval. They include moving money, entering into a contract, accepting terms and changing live data in a way that cannot be quickly undone.¹