Easy methods to create a multi-site indexing monitoring system utilizing the URL Inspection API and Analytics Edge

News Author


On January 31, Google launched the URL Inspection API, enabling you to programmatically verify the standing of URLs in bulk (primarily based on properties you’ve gotten entry to in GSC). Though you could possibly all the time examine one URL at a time in GSC to see if a URL was listed, that is the primary time you possibly can verify the indexing standing of URLs in bulk (primarily based on information instantly from Google).

The API opens up many alternatives to grasp how Google handles your URLs. Additionally, it has been nice to see the assorted search engine optimisation software suppliers transfer shortly to add the URL Inspection API to their merchandise.

Monitoring indexing standing by way of the URL Inspection API and Analytics Edge

Commonly checking the indexing standing of necessary URLs will help nip search engine optimisation issues within the bud. For instance, utilizing the API, you possibly can verify on a each day or weekly foundation whether or not your most necessary URLs are nonetheless listed. And also you positively wish to know if necessary URLs drop out of the index, whether or not it’s a results of technical issues, high quality issues, or Google’s finicky indexing system. That’s versus figuring out that drawback down the road after you’ve skilled misplaced rankings and visitors as URLs drop out of the index. And that’s when you choose up the drops shortly sufficient.

As quickly because the API was launched, I began interested by an answer to periodically bulk-check necessary URLs. As well as, I began interested by checking URLs throughout totally different websites. And going even additional, I needed to maneuver past the two,000 requests per day limitation (per property), which may be severely limiting for a lot of larger-scale websites. On this article, I’ll cowl the answer I’ve been utilizing. It’s simple to arrange and intensely cost-effective.

Analytics Edge, your search engine optimisation Swiss Military Knife

I’ve written closely about utilizing Analytics Edge for a variety of eventualities up to now. It’s an Excel add-in that lets you work with a number of APIs instantly in your spreadsheets (and supplies a ton of performance for working with that information as soon as exported). For instance, it’s an effective way to export your prime queries and touchdown pages from GSC, create Delta Reviews after main algorithm updates, and extra. And now, you should utilize Analytics Edge to bulk verify URLs by way of the URL Inspection API.

However it could actually do greater than that.

By creating a number of macros, which may be grouped, you possibly can hyperlink API duties collectively to create a robust recipe for checking the standing of many URLs throughout web sites (or GSC properties from the identical website). I’ll cowl extra about transferring past the two,000 URL restrict per day quickly.

What you possibly can accomplish by way of the URL Inspection API utilizing Analytics Edge

On this submit, you’ll learn to verify the indexing standing of your most necessary URLs, throughout totally different web sites, all at one time. As a substitute of simply checking one GSC property in bulk, we’re going to do this throughout a number of web sites routinely (with out having to set off every macro by itself).

The purpose is so that you can open a spreadsheet, click on the “Refresh all” button, and have Analytics Edge verify all of your prime URLs throughout a number of websites. It could prevent a ton of time, whereas additionally offering crucial intelligence about necessary URLs which can be now not listed.

Overcoming the two,000 requests per day restrict

You may solely course of 2,000 requests per day by way of the URL Inspection API (per GSC property). However there’s an necessary distinction between “web site” and “GSC property.” Bear in mind, you possibly can arrange a number of GSC properties per web site by including directories and subdomains, which may give you extra information (as a result of the reporting will deal with simply these directories or subdomains).

For our state of affairs as we speak, having directories or subdomains arrange as properties will allow you to course of 2,000 requests per day for every of these properties. Which means you possibly can verify many extra URLs per website when you have these extra properties arrange. And utilizing Analytics Edge, you possibly can verify the indexing standing of URLs throughout these properties in a single shot (utilizing the method on this tutorial).

Should you don’t must verify a number of web sites, however need extra requests for the one website you handle, then this could nonetheless make it easier to get extra information. So arrange your prime directories and/or subdomains. It’s simple, and extra information awaits.   

Tutorial: Easy methods to arrange a multi-site indexing monitoring system utilizing the URL Inspection API and Analytics Edge

Step 1. Obtain and set up the Analytics Edge core add-in. I’ve coated arrange Analytics Edge a number of occasions in earlier posts, and you’ll verify these posts to learn to do it. You ought to be up and operating very quickly. As soon as you put in the core add-in, you possibly can shortly add the Google Search Console connector. You will have each to perform what I’m overlaying on this submit. License-wise, there are free trials for each the core add-in and the GSC connector. After which, it’s $99 per yr for the core add-in and $50 per yr for the GSC connector.

Step 2. Now it’s time to hook up with your Google account that has entry to the GSC properties you wish to monitor. In Excel, click on Analytics Edge within the prime menu. Subsequent, click on Google Search. Then Accounts within the dropdown menu. Undergo the method of connecting your Google account and authorizing entry. This could solely take just a few seconds.

Step 3. Create a worksheet that accommodates the highest URLs from the primary web site you wish to monitor. You may embody as much as 2,000 URLs per worksheet. For this tutorial, I might solely add the highest 10 or 20 URLs for the location (so that you don’t have to attend too lengthy for the API to course of your requests whereas constructing the system). After which, you possibly can add as much as 2,000 URLs per worksheet. Ensure that the column has a header. I might use “web page” because the header on this instance so you possibly can comply with alongside. You may title the worksheet no matter you need, however I used “site1” for this tutorial. You could possibly title every worksheet primarily based on the location title.

Step 4. Subsequent, we’ll create our first macro to verify the URLs by way of the URL Inspection API. We’ll get to the second website after that (and then you definately’ll be capable of run all of the macros at one time). However for now, let’s deal with utilizing the URL Inspection API to verify indexing for the primary listing of URLs. Analytics Edge supplies a ton of performance you should utilize when crafting macros. You may discover the assorted menus to see every thing you possibly can accomplish.

Step 5. Whenever you create a macro in Analytics Edge, you mix a sequence of directions that can run collectively (like a recipe). For our functions, we wish to first learn within the listing of URLs from our worksheet, then use the URL Inspection API to verify these URLs, after which write the outcomes to a brand new worksheet (so you possibly can analyze the info).

Step 6. Within the Analytics Edge menu, click on the File menu, after which Learn Worksheet. It will pull the whole listing of URLs you’ve gotten in that worksheet. Within the dialog field, the workbook would be the present workbook, and the worksheet ought to be the present worksheet. You shouldn’t have to alter something. Simply click on OK to learn the worksheet. Additionally, discover how the Activity Pane in Analytics Edge provides every a part of our macro as we execute them. After every step, you will note the brand new instruction added to the macro.

Step 7. Analytics Edge creates a short lived worksheet named “Step Outcomes” containing the listing of the URLs you’ll course of with the URL Inspection API. Subsequent, let’s use the API. Within the Analytics Edge menu, click on Google Search, after which URL Inspection. Within the dialog field, choose the Account you arrange earlier after which the GSC property you wish to work with. Then you should inform Analytics Edge the place the URLs are within the worksheet by including the column title to the URL subject. So as to add the column title, encompass the title with brackets. So, when you used “web page” as I did earlier, add [page] within the subject. Then click on OK.

Step 8. Subsequent, let the URL Inspection API course of all of the requests (which shouldn’t take too lengthy since we’re solely testing a handful of URLs). As soon as that’s completed, the Step Outcomes worksheet will include a pattern of the outcomes highlighted in inexperienced. Subsequent, let’s write the outcomes to a brand new worksheet the place you possibly can analyze and work with the info.

Step 9. Write to Worksheet – Within the Analytics Edge menu, click on the File menu after which “Write to Worksheet.” Give your new worksheet a reputation like “site1 Outcomes” and click on OK. You now have a brand new worksheet containing the URL Inspection outcomes for the listing of URLs you wish to monitor.

Step 10. We wish to verify a number of GSC properties in a single fell swoop (throughout totally different web sites). So, let’s create a second macro to do this (after which we’ll run them collectively by way of Analytics Edge).

Step 11. Create a brand new worksheet with URLs from a second web site (or second GSC property for a similar website you ran earlier). You may title this worksheet “site2” for this tutorial. And once more, you possibly can title it by website in order for you sooner or later. Add the highest URLs from the second website you wish to monitor. Once more, I might add solely 10 to twenty URLs. You should use “web page” because the column title once more.

Step 12. Create a brand new macro that can work with this second listing of URLs. The primary macro is already arrange and can work with the primary worksheet of URLs. This second macro will use the brand new listing of URLs. Within the Activity Pane for Analytics Edge, you will note your present macro. Click on the dropdown and choose “new macro.” Identify is no matter you need. I referred to as it “Test Website 2”. Additionally, if the Activity Pane isn’t exhibiting for some purpose, you possibly can reveal it by clicking the Analytics Edge menu after which Activity Pane on the left aspect. It’s a big icon subsequent to the File menu.

Step 13. Subsequent, undergo the identical course of we did earlier to create the directions for studying the worksheet, utilizing the URL Inspection API, and writing the outcomes to a brand new worksheet named “website 2 Outcomes”. You’ll have two macros that course of URLs from two totally different worksheets whenever you’re completed. And every macro writes the outcomes to a contemporary worksheet to investigate and work with the info.

Step 14. Easy methods to monitor the URLs recurrently – Now that you simply’ve created the 2 macros, you possibly can all the time open the spreadsheet and run them at one time (to verify the URLs you wish to monitor throughout GSC properties). To do that, click on the Analytics Edge menu and click on the “Refresh All” button within the higher left nook. Analytics Edge will run each macros (in alphabetical order). As soon as the macros are full, you’ll have contemporary information to investigate. Save every spreadsheet with a brand new title so you possibly can monitor your outcomes over time.

Congratulations! You now have a multisite URL Inspection monitoring system.

You may add extra websites to watch by including extra macros to the spreadsheet. And once more, when you have a big website, then including a number of directories or subdomains as properties in GSC will allow you to maneuver past the two,000 requests per day restrict. So, you could possibly add a number of GSC properties (from the identical website) versus checking totally different websites.

Abstract – Begin with a easy URL Inspection monitoring system and increase from there

Able to create your individual system primarily based on this tutorial? Then obtain Analytics Edge (the core add-in and the Google Search Console connector) and get a easy instance working. When you do, then increase your efforts by including extra GSC properties.

I believe you’ll dig working with Analytics Edge, how versatile it may be, how briskly it could actually work, and so forth. It’s an effective way to watch necessary URLs (throughout websites or properties). And be happy to ping me on Twitter when you have any questions whereas working by means of your first macros! Comfortable monitoring.


Opinions expressed on this article are these of the visitor writer and never essentially Search Engine Land. Workers authors are listed right here.


New on Search Engine Land

About The Writer

Glenn Gabe is an search engine optimisation advisor at G-Squared Interactive (GSQi) and has over 24 years of expertise in digital advertising and marketing. Glenn at present helps purchasers with algorithm replace restoration, technical search engine optimisation audits, search engine optimisation for web site redesigns and migrations, and search engine optimisation coaching. Throughout his profession, Glenn has helped firms throughout a variety of industries together with publishing, e-commerce, well being and medical, finance, startups, army, schooling, non-profits, actual property, affiliate internet marketing, and extra.