Tools are a vital part of a Marketer’s arsenal, the SEO industry – in particular – is pretty nuts about tools. For the last two years, I’ve focused more on the effective application and teaching of how to get the most from these tools – it’s something I’ve become quite passionate about.
However, one issue I repeatedly see is that people don’t often focus on is what exactly some of these tools actually report on. Rank trackers, backlink checkers and search intelligence tools all do different-yet-similar jobs – many of which create their own scoring systems to help people out.
In the search marketing space, we live in a world where a number of the metrics we (and the tool providers) use sound like one thing, but mean another. Some of my “favourites” are Clicks vs Visits, PageRank Vs Domain Authority, Rankings Vs Avg Ranking.
But the MOST damaging confusion I see is “traffic” (as measured by Search Intelligence tools) and Organic Traffic as reported in Google Analytics.
For example, in this graph we can see WarnerBros.com’s domain dropping 60% from one week to the next:
What does this mean? Well, a wrong assumption would be that they lost 60% of organic traffic overnight (although there’s a strong chance they lost some traffic). This site became 60% less visible. A crucial difference.
To help you understand the difference between actual traffic and an approximation of visibility, I’ll take two of my favourite Search Intelligence platforms to explain what exactly they report on AND give some examples to show you.
What Do these Tools Track?
Search intelligence tools are growing in their sophistication, they’re aiming to help report on backlinks, keyword research, content analysis and much, much more. However, the main selling point of these platforms is often the way they profile “search visibility”.
“Search visibility” is broadly how many times a website appears within search results for a predefined list of keywords – and at what position. The more times you appear and the closer you are to position 1, the more visible you are.
This is really, really valuable information as it can provide information on your competition, the marketplace and more importantly tracking search results/keywords you hadn’t thought of yet.
I will look at SISTRIX and SEMrush as part of this, I use these the most myself and find they’re very easy to use easily in a successful marketing campaign.
SISTRIX Visibility Index
Full disclosure, I’m a SISTRIX certified trainer, so I am involved with the platform heavily – I’ve explained the visibility index to many, many people. But here it is in their own words:
For the calculation of the Visibility Index, the top 100 search positions for 1,000,000 keywords, or keyword-combinations (search terms), are evaluated every day…
Once the Keyword results are gathered, the results are weighted according to the position and search volume for the particular keyword. This means that position 10 for a very traffic-strong keyword like “apple” corresponds to a higher value than 1st place for a less-searched search term like “display problem iphone x”. When we weight the positions we also take the different click rates of the ranking positions into account.
The Visibility Index aims to represent the actual click behaviour of the Google-user for each of the ranking positions, as best as possible.
This 1 million strong index has been 90% consistent since 2010 (in the UK) which means a consistent benchmark for nearly 10 years – very powerful stuff. They have 30mil plus keywords in their tool kit, but that 1million is carefully selected sample to ensure the index is stable AND seasonality is removed.
SEMrush Domain Analytics (Estimated Organic Traffic)
SEMrush has a comparable product in its domain overview, where a similar appraisal of a website is made using ranking data. Again, for full disclosure, I’ve worked with SEMrush on a number of webinars, blogs & recently their Global Marketing Day.
Domain Overview presents a range of overview metrics about a domain’s online visibility so you can quickly get a full picture of a website’s presence. You’ll see a domain’s organic traffic… A quick overview of any domain’s overall search engine presence.
In the Organic Search section, you will find… keywords that a domain is ranking for in Google’s top 100 organic search results, keywords ranking distribution, and the domain’s organic competitors.
SEMrush has a larger index (24m terms on desktop in the UK), but not as much of a focus on the consistent benchmark.
The Difference between Visibility Data and Organic Traffic
I have explained the difference to many people over the years, but in a bid to make it easier to SEE the difference, I’ve conducted some tests. I will use SISTRIX, SEMrush and Google analytics to compare Visibility Index, Online Visibility and Organic Traffic.
One decision I have made is to obscure which tool is which in the results I’m publishing here. I have no intention to state which is best (there is no objective way to do so), merely to help educate how to use and interpret the data.
Test Conditions
The test was simple, pull the same period of data from each data source with the following in mind:
- Google “organic” data from Google Analytics
- Search Visibility data from the main territory the website is visible in
- Anonymised data – only broad data to illustrate niche/industry
- Data provider’s source obscured
Then the data was graphed so we can easily compare how each differs.
Test Data
I am used to telling people that this data is often different, but I wasn’t prepared for just how different it so often would be.
If we assume that we want the Blue & Yellow lines to be as close to the Blue as possible (or at least following the same trend), you can really see some points where neither Tool is all that close.
(I’ve also converted the slides into a GIF for easier sharing)
To help outline just how different each of these data series were, I also calculated each of the tools and the correlation to actual organic traffic.
If you’re not used to correlations, +1 means a perfect correlation (the data is the same) and -1 is a total negative correlation. We don’t expect anything to be perfect in a positive or negative sense, so decimal places illustrated the scale between +1 and -1.
What is encouraging is that the vast majority of the data is a positively correlating – more right than wrong, representing a reasonably strong correlation on average. However, there are a number of examples in there where the correlations are moderate or weak – 1 where the correlation is strongly negative.
I won’t go into much detail about each example site, except to see that example 7 is a well-established website which has a pretty strong level of traffic/rankings.
The Dangers of Using this Data Wrongly
If you spend enough time looking at the charts you’ll see how far off (in many places) the tools are from the actual data, it makes this feel pretty unreliable used in this way. Taking a look at the correlations, anything below 0.4 starts to feel like a gamble and anything below 0.2 is almost worse than just guessing in your decision making.
Assuming a site is doing well (or not) without checking more than one data source (i.e. one tool or platform) is asking for trouble as the margin for error is quite high. Decisions which are likely to lead to financial outlay or investment without correct inspection will be highly risky.
But this misses the point!
Benchmarking, Measurements & Doing Things Right
What these tools excel in is benchmarking performance, from today’s performance against yesterday’s or from your performance today against your competitor’s today.
Everything in this sense is relative, without comparisons to a previous period are relatively pointless. What’s worth a word of caution is SEMrush referring to ‘traffic’ with their score – which is where a misunderstanding could easily slip in for the uninitiated.
Some ways to get the most value out of these tools:
- Comparing performance pre & post-migration/launch
- Comparing your performance to your competition
- Identifying potential competition
- Identifying fluctuation within search results (algorithm update)
- Keyword research/discovery
By seeing what Google is telling you (via the Search Results themselves), you get to bypass overly complex scoring systems which are too abstract/confusing to be all that useful in the long-term.
Conclusion
It is easy to see where this confusion is coming from, “visibility” as a concept for those who are not familiar with search marketing is tricky to get to grasp – trying to explain it in detail isn’t always easy either!
It is worth mentioning that neither tool is professing to show actual traffic here – this is not the first data/investigation which highlights how risky this wrong assumption is.
Using these scoring systems to benchmark/compare you will find the concept easier to understand & interpret and is far, far safer.
An over-reliance on this data when reporting to a senior stakeholder or client is also risky at times – if you fail to explain what the data is telling you, that is.
