Let’s talk→

Google Publishes Guide to Allowing Google-Agent Traffic

Jen Cannizzo, founder of GrowthopediaBy Jen Cannizzo··3 min read
Google Publishes Guide to Allowing Google-Agent Traffic

Key takeaway

Google added a guide on November 17, 2026 explaining how to let Google-Agent, its user-requested AI agents, through your CDN, firewall and bot protection.

On November 17, 2026, Google published a new technical guide explaining how to make sure Google-Agent, the user agent for its AI agents that browse the web on a user's behalf, can reach your website without being blocked by your CDN, firewall or bot-detection service.

What changed

Google's crawling documentation changelog lists the November 17 entry as "Added guidance on allowing Google-Agent". The new page, Allow Google-Agent traffic, explains how to let these requests through while keeping your existing security controls in place.

Google-Agent isn't new. Google's user-triggered fetchers page describes Google-Agent as being "used by agents hosted on Google infrastructure to navigate the web and perform actions upon user request." What's new is a practical guide to allowing it. The page covers three ways Google-Agent identifies itself:

  • User agent. The User-Agent header contains "Google-Agent." Google lists both a mobile and a desktop user agent string.
  • IP addresses. Requests come from the IP ranges Google publishes in its user-triggered-agents.json file.
  • Signed requests. Google says it is experimenting with Web Bot Auth, using the identity https://agent.bot.goog, so some requests are signed. Those signed requests use HTTP Message Signatures (RFC 9421) and include Signature-Agent, Signature and Signature-Input headers.

Google also notes on its user-triggered fetchers page that, because these fetches are requested by a user, these fetchers generally ignore robots.txt rules. So if you want to control this traffic, robots.txt isn't the lever. Your CDN, firewall and bot rules are.

Who it affects

This matters most if your site sits behind a CDN, web application firewall or bot-protection service, which is common for ecommerce stores, booking sites and any business that has tightened security against scrapers. It also matters if your customers might use an AI agent to search, compare, book or buy on your site. If those requests get blocked or hit a CAPTCHA, the agent can't finish the task for the person who asked it to.

If you run a simple brochure site with no special bot rules, there may be nothing to change. It's still worth a quick check with whoever manages your hosting.

Want to know if AI recommends your business?✦ Free AI check

What to do now

Google's guide recommends using your provider's built-in verification for Google-Agent if it has one. If it doesn't, Google suggests verifying the user agent string, checking the source IP against the published JSON file, verifying signatures when they're present, and skipping bot challenges for verified requests. The troubleshooting section adds a few things I'd check with your developer or host:

  • Rule order. Make sure your allow rules run before broad ASN blocks, managed bot rules or any policy that blocks AI agents.
  • Challenges, not just blocks. Google says to skip CAPTCHAs and JavaScript challenges for verified requests, not only hard HTTP blocks.
  • The whole journey. Allow rules should cover search, filtering, detail pages, cart and booking steps, plus the XHR, GraphQL or API endpoints behind them.
  • Rate limits. Exempt or account for verified Google-Agent IP ranges in per-IP rate limits.
  • Keep IP lists current. Keep your IP rules in sync with Google's JSON file rather than hard-coding a list once.
  • Proxies and load balancers. Make sure the client IP, User-Agent and signature headers are passed through so your rules can see them.

If you'd like a second pair of eyes on this, it's the kind of thing I look at in technical SEO work and a technical SEO audit. If you're already digging through your logs, my post on how Google says to spot Search traffic in analytics and logs may help too.

Was this helpful?

Jen Cannizzo

Written by

Jen Cannizzo

Founder of Growthopedia, an Albany, NY SEO and digital marketing consultancy. She has run technical SEO, content and reporting programs for enterprise brands and local Capital Region businesses.

About Jen →

Questions, answered

Is Google-Agent the same as Googlebot?

No. Google lists Google-Agent among its user-triggered fetchers, which are started by a user to perform a task within a Google product. Googlebot is Google's crawler for Search. They use different user agent strings, and Google-Agent's IP ranges are published in a separate file, user-triggered-agents.json.

Can I block Google-Agent with robots.txt?

Google says user-triggered fetchers generally ignore robots.txt rules because the fetch was requested by a user. If you want to allow or restrict this traffic, that happens in your CDN, firewall or bot-detection settings, which is what the new guide is about.

How do I know a request really came from Google-Agent?

Don't rely on the user agent alone, since anyone can copy it. Google's guide suggests checking the source IP against its published IP range file and, where present, verifying the request signature using the public keys Google publishes for agent.bot.goog.

If you're not sure whether your security settings are turning away legitimate agent traffic, feel free to get in touch and I'll take a look.

Need help with this?

01

SEO Services

Search Engine Optimization (SEO) is the process of improving your website’s structure, content, and authority so it ranks higher in Google search resu

→
02

AI Search Optimization (GEO)

Generative Engine Optimization (GEO) is the next evolution of SEO , built to help brands appear inside AI Overviews , Search Generative Experience (SG

→

Field notes

Keep reading

All posts
Google's Ask Maps Can Now Call Local Businesses to Find Appointments
Read next

Google's Ask Maps Can Now Call Local Businesses to Find Appointments

Start a project

Let’s make you findable.

Tell me what you’re working on. I read every message and reply with honest next steps.

  • Free strategy call, no pressure
  • Honest, data-backed recommendations
  • Reply within one business day

Tell me about your business

  • Honest recommendations
  • No obligation
  • Reply within 1 business day

5.0 on Google · No spam, ever