Robots Exclusion Checker
Robots Exclusion Checker is designed to visually indicate whether any robots exclusions are preventing your page from being crawled or indexed by Search Engines.
The extension reports on 3 elements:
2. Meta Robots in the HTML <head>
3. X-robots-tag in the HTTP header response
If the extension detects that the current URL is affected by an exclusion, the robot icon will turn Red. If nothing is detected the icon will remain turn Green. The extension has been designed primarily for ‘googlebot’ & ‘googlebot-news’ user-agents but we'll be adding more in the future.
This tool will be useful for anyone working in Search Engine Optimisation (SEO) or digital marketing as it gives a nice visual indication if the page is being blocked by robots.txt (many existing extensions don’t flag this). Crawl or indexation issues have a direct bearing on how well your website performs in organic results so this extension should be part of your developer toolkit for Google Chrome.
Found a bug or want to make a suggestion? Please email email@example.com