I’ll be honest, this is my least favorite Google Search Console error.
You spend hours creating content on your website (not to mention the research prior to writing, and maintenance after publishing) and then Google hits you with this:
Come on Google, give a marketing team a break here!
Why would Google crawl but not index content? And what do you do about it? We’ll answer these questions and more in this article.
Google flags the crawled but not indexed error when it has crawled a page but has decided not to index the page in search results.
Basically, Google is saying “We see the page, but we’re not showing in search results.” This is similar to “Discovered-currently not indexed” which means Google knows of the page, but they have not even crawled it yet.
Google at least does you the favor of showing you which pages they have decided not to index, along with some information that could show you why.
How to find pages that are “Crawled - currently not indexed”
To find your pages that are crawled but not indexed, first head to Google Search Console and click on Pages in the lefthand side menu. That will bring you here:
You’ll be greeted with a chart showing the number of indexed and not indexed urls on your site, with a table below that looks like this:
Note the ‘Crawled- currently not indexed’ table row. Click on it, this will open up a chart showing the number of pages that are crawled but not indexed, how that has changed over time, as well as a list of URLs:
Clicking on a URL will open up a side panel that allows you to inspect the URL further to review data from Google on when they last crawled it, and any reasons why Google decided not to index it.
Why would Google do this though?
So why would Google crawl but not bother indexing a page? Google tends to be mum on specifics, though there are a few common reasons that this can happen.
Google is on a mission to provide the best answer to every question, in order to do that Google needs to ensure that the search results it shows are high quality.
Google has ramped up its efforts recently to crack down on low-effort content, meaning content auto-generated by AI or hastily copy and pasted from aa variety of sources (see our analysis on the helpful content update for more info there). As a result, if Google thinks your page is a bit thin compared to other content that ranks for the same queries, you may find your page being excluded from the index.
In SEO we call pages that are not linked to from anywhere ‘orphan pages’. They’re forgotten children on your website, but unlike a Dickens novel these pages won’t win over the hearts and minds of your customers - they won’t be seen at all!
When crawling your website, Google pays close attention to how far it needs to go in order to get to any given page. Your homepage is the most important page on your website, so the further you go out from your homepage the less important Google thinks the content is. Pages that are not linked to anywhere are the least important, at least to Google, and could not be indexed as a result.
In SEO we have a concept called ‘search intent’, it sounds technical but really it’s just a way of guessing what the intent behind the search is. In other words, what is the searcher looking for/looking to do?
The example I always give is if I am Googling the temperature to cook chicken breast, I do not want 10,000 word dissertation on chicken. I want to know it’s 165 degrees Fahrenheit so that I do not give myself food poisoning.
If your content does not match the search intent, Google may de-prioritize indexing it. We’ve written more on search intent so check that article out for more info.
As time goes on its easy to let your website get out of hand and start doubling up on blog topics.
When you double up on topics, Google will try to pick one page to show in search results. This does you a favor actually, otherwise you’d be at risk of keyword cannibalization. Still, it’s not a great thing to make a habit of as it will lead to a bloated website that becomes a bit of a mess to Google and your users.
The best thing you can do is keep track of your content, mapping content to keywords and pruning your content regularly to ensure your website stays tidy and organized.
There are a variety of reasons that Google may crawl but not index a page, so now how do you fix that? We’ve got a few best practices we recommend so let’s dive in further.
First and foremost, focus on content quality.
As we said above, Google is hyper-focusing on content quality so the more you can align with Google’s goals the better you will be rewarded.
By content quality we mean two things:
Make sure that your content comprehensively answers the question behind the search, properly addresses it, and serves your reader. If you do those things, there’s a good chance Google will decide to index it.
A quick way to signal to Google that your content is important is to link to it from an important page.
Often I suggest linking pillar pages from the footer navigation (strong signal to Google that they are important), and using these pillar pages to link out to related blog posts or content pieces. If your page that is not being indexed aligns with one of these existing important pages, a link to it could signal to Google that it is important to your website and worth being indexed.
There’s always the option to manually request Google to re-crawl the page which could do the trick as well.
This is not an immediate solution, all it does is request Google to put the URL in queue to be crawled, how long that queue is or where you stand it in is a mystery. Still, it’s better than nothing if you’ve already exhausted your other options.
Get the same strategy framework we teach every single client. Follow these 4 steps to outsmart your competitors on Google and rank your website higher than ever.