Eric Wu open-sourced his SEO tool that bulk checks IPs to validate Googlebot IPs. 🤖 Why is this important?
“Often there are various requests by spoofed user-agents pretending to be official search engine crawlers like Googlebot. In order to have an accurate understanding of the site’s crawl rate, we want to verify the IP address of the various crawlers.”
