Ever since Panda 2.5 struck in late September and mid October, I’ve noticed more e-mails, tweets and posts that express concern over the Google Webmaster Tools Crawl Error reports.
Many people are wondering if this report has something to do with their sudden drop in the rankings. Others have been bothered by this for awhile, but are not sure how to handle it.
I don’t know about you, but I have tons of crawl errors in my reports, but they aren’t errors I’ve generated.
They’re caused by people who have attempted to link to my site and misspelled the URL, thus generating the error in my Google Webmaster Tools reports.
And if the person is using some kind of automated system or CMS, then it often results in multiple errors because the URLs have been dynamically generated incorrectly over and over again on different pages.
Here’s why you are seeing these errors…
As the Google spider roams the web and finds a link that points to your website URL, it tries to crawl that link.
If there is no content there, the Google Webmaster Tools returns a 404 and shows this error in your Crawl Error report.
The good news is Google tells us on the Webmaster Central blog that there’s no need to worry about errors that don’t exist on your own site. They state and I quote…
We don’t know which URLs are important to you vs. which are supposed to 404, so we show you all the 404s we found on your site and let you decide which, if any, require your attention.
What’s a bit misleading about that statement is where they say “…we show you all the 404s we’ve found on your site.” Well, the problem is these errors are not physically on our sites. So I can see why that report causes confusion.
Nevertheless, it’s a good idea to stay on top of this, but no need to panic over broken URLs that don’t exist on your site. Just fix the internal errors that you can control.
If you notice a common misspelling of a certain URL in your reports, you could create a 301 redirect with your host to make sure that link gets redirected to the proper page.
I personally have found no reason to do this since my errors seem to be all over the place in terms of spelling. Nevertheless, you may find this works for you.
So if you’re worried about this report impacting your site’s reputation with Google in a negative way, worry no more.
Google clearly states this is nothing to worry about and you can (according to their blog) safely ignore the 404s generated by external sites.