1. Make pages discoverable and accessible
- Check
- Check robots rules, the HTTP response, readable page text, internal links and language references together. Permission in robots.txt is insufficient if a firewall blocks access.
- Suitable evidence
- The specific URL, crawler tested, time and content actually received. For multilingual sites, check each language URL separately.
- Limit of the finding
- A successful fetch proves neither indexing nor a future citation.