We have two redundant facilities: east and south. Often we’ll switch
100% of traffic to one site or the other. We noticed an abnormally
high volume of 404 errors on 8/5. Googlebot spidered cached pages on
our east site, which at the time was down for maintenance. Our south
site at the time was fully functional. However, Googlebot continued
to spider the east site while it was down. How does Googlebot cache
DNS and/or use TTL?
On 8/16 the east site was down for four hours for network
maintenance. I noticed 2,800 or so errors generated by Google on
8/16. Was that a result of the same functional use of caching and TTL
by Googlebot?
Andrea