When you build websites with thousands (or tens of thousands) of pages, you have to know how to get indexed on Google as fast as possible.
There are lots of options you can take to encourage Google to crawl your site faster and get as many pages as you possibly can indexed. More pages indexed means more chances to rank for long tail keywords, so we want to make sure Google knows about all pages.
Here are some of the ways you can speed up the indexing process to get indexed on Google faster.
How to speed up Google indexing
I use as many options as I can to index pages faster, because the more signals you give to Google that this URL is worth indexing, the more likely they are to index it.
Create more internal links
When the Googlebot crawls your site, it follows links to find more pages. So it should come as no surprise that internal links are a great way to show Google where all of your pages are, and make it easy for them to be crawled.
As a general rule of thumb, every page on your site should be no more than 3 links away from the homepage. I try to make each page 2 links from the homepage where possible. This makes it as efficient as possible for the crawler to find more pages fast.
I also use breadcrumbs on my blog posts, which maps this hierarchy back to the homepage.
If you use WordPress, Link Whisper is a great plugin to find and suggest topical internal links. Since I’m not using WP, I built my own engine using Python.
For many of the same reasons as internal links, external links are a big sign to Google that this page is worth indexing. Not only do they let Google discover new URLs but they also show that the page was worth linking to, and this is a suggestion that it’s also worth indexing.
However, be careful of some indexing tools which use spammy backlinks in order to get your pages indexed. It’s still a prerequisite that the backlinks be good quality and from an authoritative site.
Create an XML sitemap
WordPress and most other CMS options have sitemap generation built in. For large programmatic sites like the type I build, these are very important to submit to Google Search Console, Bing Webmaster Tools, and others to let the search engines know about this large volume of pages that exist.
You can have sitemaps of up to 50,000 URLs in each, but for best indexing results you should never have more than 10,000 URLs. This has been the sweet spot for me in getting sitemap URLs crawled and indexed quickly.
Use the Google Indexing API
The Google Indexing API is supposedly only able to be used with JobPosting or BroadcastEvent type content. However, a lot of people, myself included, have been using this to notify Google and get pages indexed that are not either of these content types.
In fact, I built URL Monitor using this API for indexing. Not only can you submit 400+ URLs per day, but it also monitors the indexing status of your site daily to alert you to any potential issues.
URL Monitor takes care of all the technical aspects of setting up the Google Indexing API and tracking your URLs. This is how to get indexed on Google faster without any coding needed.
In as little as 48 hours your pages can be indexed. Now, because it’s using the API, there is a maximum of 200 URLs per day that can be submitted. If you have huge sites like me, it will take a long time to worth through that list and get them all indexed.
Rank Math Instant Indexing
In WordPress, the RankMath SEO plugin can be used to submit URLs to the IndexNow service. It's not as fast as the Indexing API, but it's a good option - especially for other search engines like Bing.
This feature will be added to URL Monitor soon.
Before I built my own tool, I used IndexMeNow a lot.
It's much more expensive, as you pay per URL, but it doesn't rely on the Google Indexing API so you can submit as many URLs as you want.
I used to use it to focus on my archive pages which contain a lot of links to inner blog posts and articles to encourage the Google crawler to find and index those pages.
Turn on Cloudflare Crawler Hints
While they don’t help you to get indexed on Google faster, Cloudflare Crawler Hints can help you accelerate the indexing process in search engines like Bing and Yandex. And if your website receives a significant amount of traffic from these search engines, you should consider using this simple feature.
To turn on the option, just navigate to Cache > Configuration in your Cloudflare account and turn on the toggle button for Crawler Hints.
Changing your title tag
If you use all of these methods and still find that your URL is not being indexed (assuming that your page is objectively worth being indexed) then one tip that hasn’t been shared by anybody else that works well for me is to simply change your title tag slightly and resubmit it.
I use this a lot in combination with IndexMeNow because they refund your credit if the URL does not get indexed. So I change the title tag, resubmit it, and most times I do this the page gets indexed.
You could also experiment with slightly rewording your post introduction to make slight changes to encourage it to get indexed.
All of these tips can be used to get indexed on Google faster, but they are all options that you can only use if you have high-quality pages to begin with.
No tactics will work to get your pages crawled and indexed if the content is not good enough. For programmatic SEO sites this means you need to spend as much time on your database and your template as you can to ensure it provides a helpful and useful experience for your site users.
Once you have that, combined with some of the methods listed in this blog post, you’ll know how to get indexed by Google and can apply it to your sites.
If you’re new to the topic of programmatic SEO, I will suggest going through this blog post about the topic.
Link to this page
I can haz links? If you can, do me a solid and link to this article if you mention it in your blog post or newsletter.