input license here

List of Googlebot IP addresses released














Google has published a complete list of IP addresses that Google uses to crawl and access your website under Googlebot user agents. Google said if you do not want to use reverse DNS other than those types of methods to verify Googlebot, you “can identify Googlebot by IP address by matching the crawler IP address with Googlebot's IP address list . ”


Why do I need this. If you believe that someone is crawling your site, pretending to be Googlebot, and you want to verify that blocking this fake Googlebot crawler is not actually Google, you can use command line method or automatic IP list method .


IP list. You can access the complete list of Googlebot IP addresses here in this JSON file , note that Google may update this list, so you should check the file possibly daily. You can also access all other Google crawlers matching the crawler IP address based on Full list of Google IP addresses . Again, these IP addresses can change over time, so you should build in your mechanism a way to update the IP address on your part.


Why do we care. Often websites can slow down and even potentially go offline due to fake bots crawling and manipulating the site. You rarely want to block Google from crawling your site as that can lead to indexing and ranking problems in Google Search. So knowing which bots are actually Google and which aren't, can help you decide which fake bots to block from your site.



There are also third-party services, like Cloudflare and others that help you manage this.
















.
Related Posts
Diệp Quân
Nguyen Manh Cuong is the author and founder of the vmwareplayerfree blog. With over 14 years of experience in Online Marketing, he now runs a number of successful websites, and occasionally shares his experience & knowledge on this blog.
SHARE

Related Posts

Subscribe to get free updates

Post a Comment

Sticky