The SEOLinkMap Bot
How our crawler identifies itself, what it does, and how to control or report it
How to recognize it
Every request SEOLinkMap makes to your site carries this user agent:
SEOLinkMap/1.0 (+https://seolinkmap.com/bot)
The name to use in robots.txt is SEOLinkMap. If you see that string in your logs, the request came from us and this page explains why.
What it does
SEOLinkMap makes two kinds of request, and they are not the same thing.
Site crawls
When someone adds a domain to a SEOLinkMap account, we crawl that site to build its link map, page inventory and content metrics. This is the repeated, multi-page activity you are most likely to notice. It reads publicly reachable HTML only, follows internal links, and reads your sitemap when you publish one.
Single page analysis
Our free public tool fetches one page at a time when a person asks for that specific URL to be analyzed. It is a single request for a single page, made on demand, the way a browser would. It does not follow links onward and it does not come back on its own.
What it reads
SEOLinkMap requests publicly reachable HTML pages over HTTP and HTTPS on the standard ports. It reads titles, headings, meta descriptions, meta robots directives, link structure and body text.
It does not submit forms, does not attempt to log in, does not accept or carry credentials, and does not request pages on private or internal networks.
Controlling it
SEOLinkMap honors robots.txt and the standard exclusion directives. Changes take effect the next time we read your robots.txt.
Block it entirely
User-agent: SEOLinkMap Disallow: /
Block part of your site
User-agent: SEOLinkMap Disallow: /checkout/ Disallow: /account/
Slow it down
Set a delay in seconds between our requests:
User-agent: SEOLinkMap Crawl-delay: 10
A rule written for User-agent: * applies to us as well when no SEOLinkMap-specific group is present.
Reporting a problem
If SEOLinkMap is taking more of your server than it should, ignoring a rule you have set, or requesting something it has no business requesting, tell us and we will stop it at our end:
It helps to include:
- Your domain
- The dates and times, with the time zone
- The source IP addresses you saw
- A few of the requested URLs, or a short log excerpt
We can block a domain from our side, which stops both the crawler and the public analysis tool from requesting it again, regardless of what anyone asks us for. You do not need an account with us to ask for that.
Contact
For anything that is not a complaint about the bot, write to support@seolinkmap.com.