Find the Pages Google Has Not Indexed
Indexing Gap Analysis, Free With Search Console
We crawl your site, ask Google's URL Inspection API about every page we find, and list each one Google has not indexed, with the reason Google gives. It runs on the free plan once you connect Search Console.
What Counts as an Indexing Gap
An indexing gap is a page on your site that is meant to be in Google's index and is not. Search Console's page indexing report covers the URLs Google already knows about, so it cannot list a page Google has never come across.
We start from your side instead. Our crawler builds the list of your pages, and each one is checked with Google directly, so a page Google has never seen shows up next to the ones it has seen and set aside.
How the Check Works
Four steps, and after the second one they run on their own
We crawl your site
The crawler follows your internal links and reads the sitemaps listed in your robots.txt. Free projects hold up to 500 pages and are crawled monthly. Professional, at $10 a month, allows 10,000 pages per project and daily crawls.
You connect Search Console
Sign in with a Google account that has access to the site's property and pick it. Search Console and GA4 sync are included on every plan, the free one too.
Google is asked about every page
Once a day our sync sends crawled URLs to Google's URL Inspection API and stores each answer. Google allows about 2,000 inspections per property per day, so pages never checked go first and every page is checked again after seven days.
The gap is listed
Every crawled page whose latest answer is anything other than "Submitted and indexed" goes on the list. When nothing is missing, the tab says every URL is indexed.
What Each Row Tells You
Each row is one URL with its last inspection date and three answers from Google: coverage state, indexing state and robots.txt state. Read coverage state first, because it says why the page is out.
These are the coverage states that show up most, in Google's own words.
- URL is unknown to Google
- Google has no record of the page. It needs a link from a page Google already crawls, or a place in a sitemap you submit.
- Discovered - currently not indexed
- Google knows the URL but has not fetched it yet. More internal links from pages Google visits often is the usual first move.
- Crawled - currently not indexed
- Google fetched the page and chose to leave it out. Thin or near duplicate content is the common reason.
- Duplicate, Google chose different canonical than user
- Google indexed another URL in place of this one, even though this page names itself as the canonical.
- Indexed, not submitted in sitemap
- The page is indexed but missing from your submitted sitemaps. Only "Submitted and indexed" counts as clean, so these appear too, and adding them to a submitted sitemap clears them.
A page we have not inspected yet shows as Unknown to Google until its turn comes. On a site under 2,000 pages, every page gets its first inspection in the first sync after the crawl.
Which Pages Are Left Out
Pages you have told Google to skip are left out, so the list holds only pages you want indexed. The crawler obeys robots.txt disallow rules, drops pages with a noindex robots meta tag, and folds a page whose canonical tag points to another URL on your site into the page it names.
That keeps the gap from filling up with pages that are out of the index on purpose. If a page you expected is not on the list, check those three things first.
Where the Gap Shows Up
The full list lives on the Indexing Gap tab of each project's LinkMap, as a table you can sort by URL, date or state. On the map itself, Indexing Gap is one of the metrics you can color pages by, so unindexed pages stand out among the pages that link to them.
The report builder has three Indexing Gap widgets: the full table, a single count, and a donut split into indexed, known but not indexed, and unknown to Google. A report can go to a client through a guest link, which the free plan includes.
Closing a Gap on Your Site
Most gaps close by giving Google a better path to the page. The LinkMap shows how deep each page sits and how many internal links reach it, and a page with few links in is the usual suspect.
After you add links or fix the sitemap, the page is inspected again within seven days of its last check and drops off the list once Google reports it as submitted and indexed. New links are picked up on the next crawl, monthly on free and daily on Professional.
What We Are Promising
And what we are not
What you get
Every crawled page checked against Google's own answer, rechecked every seven days, with Google's reason shown for each page that is not indexed. Pages you told Google to skip stay off the list. Free with Search Console on up to three projects of 500 pages each.
That is delivered every time.
What we do not claim
We cannot make Google index a page, and this check does not submit anything to Google. Google decides what goes in its index. We show you where it said no and the reason it gave.
Each answer is as old as its last inspection. That is normally seven days or less, and longer on a site so large that a full pass needs more than a week of Google's daily allowance.
Find Your Missing Pages
Crawl free, connect Search Console, and the list fills in on the first sync.