Linkdex New Site Crawler Out of Beta

Linkdex New Site Crawler Out of Beta

29th May 2012

It is time for some news on tools again! This time exciting news from the people over at Linkdex. The “SEO Platform” today announces their hot new site crawler comes out of beta.

The new crawler was part of a major update a few weeks ago but is now being ‘opened’ for more than just the happy few who were ‘let in’ earlier. The crawler lets you discover which content you have is under performing.  The feature actually has also been shortlisted for a European Search Award (just like State of Search!) in the category ‘Best SEO Software’.

Search operators

Linkdex adds a feature called ‘search operators’ to their tool, which works exactly like you are used to in Google (or Bing). You can do a search for “intitle:keyword” and all pages on your site that have that keyword in the title. That is handy, but Linkdex has added some more which will especially give larger sites useful information on their content.

The search operators Linkdex lets you use are:

  • intitle:keyword
  • inurl:keyword
  • inheading:keyword
  • inintanchor:keyword
  • inextanchor:keyword
  • indesc:keyword
  • infolder:keyword
  • inalt:keyword
  • inpage:keyword

This way you can filter down pages on specific topics and make re-use them for the right purposes.

Take a look at the screen below for example where we highlighted the pages on State of Search which have the word “Google” in the title and the word “change” in at least one of the headings.

Interesting is however that Linkdex for that has to hold a copy of the pages on their servers, which means they are doing the same as Google: crawling and storing your site.

More information about your pages

By crawling your site and indexing everything there Linkdex is also capable of giving you a lot more information on your pages than before. It now gives you page level information on:

  • Top 10 Ranks – Number of keywords being rank tracked in the top 10
  • Direct Natural Visits – Natural search visits entering at a specific page URL
  • Phrases Providing Visits – Number of unique phrases providing natural search visits to a specific page URL (a pages keyword reach)
  • Internal Links – Number of internal links pointing at a specific page URL
  • External Links – Number of external links pointing at a specific page URL
  • Top Internal Link Anchors – The top internal link anchors and count pointing at a specific page URL
  • Top External Link Anchors – The top external link anchors and count pointing at a specific page URL

Take for example on of the pages we showed above. Highlighting them will give us more information in the right pane, like the potential issues with them (and the option to make it a task), but also how much traffic it has gotten from search in that period of time.

And to make things complete Linkdex also added site optimization widgets, a site summary widget and an internal & external link distribution chart.

Really improve your site

We at State of Search have had the possibility to work with the functionality for a few weeks now and I must admit: it is extremely useful, and at the same time extremely scary. We knew we could do more with our fantastic content we have written over the past few years, but Linkdex now points us exactly to which content is performing and which isn’t. That makes our restructuring of the site a lot more interesting.

The new tools give you a lot more information on your site and give you actual handles to start improving your site. It really means yet another step forward in the development of the tool I believe.

Matt Roberts, the brains behind Linkdex agrees on that. He told us:

“I cut my teeth with great tools like Xenu and later Screaming Frog, but there was always key page level information missing. Like most SEO’s I had to hack this in using complex spreadsheets to get to a position where good decisions could be made. These processes were hugely time consuming, weren’t scalable and prevented you adding value in other areas.


I strongly believe that SEO’s more than being technical and good with spreadsheets, it’s also about being a tenacious, flexible and creative marketer, and this means freeing up time. So together with an exceptional development team we not only made what good SEO’s spend a long time creating with spreadsheets quick and easy, we took the feature set further. I’m really proud of V1 of the onsite crawler.”

It is interesting and good to see the tools like Linkdex, Majestic but also the other ones like Searchmetrics are constantly evolving and expanding. The data and right tools can make the industry again take important steps forward.


Written By
Bas van den Beld is an award winning Digital Marketing consultant, trainer and speaker. He is the founder of State of Digital and helps companies develop solid marketing strategies.
  • This field is for validation purposes and should be left unchanged.