Could information accessibility be the biggest threat for Google?

What’s the biggest threat for Google? Bing? Wolfram|Alpha? Facebook? Twitter? Or is it something completely different? It could be the accessibility of information on the internet. While Google is trying to index more existing (offline) information with programs where they digitize information from libraries all over the world, more new information that is created isn’t created within the ‘open’ world wide web. A new threat is forming for Google.

The internet consists of several resources of information, the world wide web is just one of them. Search engines basically try to crawl and index the world wide web, which mainly consists of a collection of HTML documents. And that’s exactly where Google reigns. But since 2000 the web is in decline, as the chart below shows. Other parts of the internet are becoming increasingly important sources for information. At the same time within the web there are always old and new obstacles that obstruct search engines form either crawling or indexing the published information.


Mobile apps

One of the most important upcoming sources of information are mobile apps. Mobile apps have one major problem for web search engines: the information they contain are not (yet) crawlable. Basically the most mobile apps at this moment are fed by information that’s also available on the web. But once the choice is made for app-only web search engines lose the possibility to access the information.

Luckily there is a solution for this problem: the mobile web. The mobile web has however some major limitations towards mobile apps. The integration of device capabilities like GPS, camera etc. is more limited than for mobile apps. In performance it heavily depends on the possibilities of HTML and CSS. And here lies the solution for Google: with HTML5 and CSS3 these possibilities can compete with mobile apps. That’s why Google is a major contributor to the development of HTML5. In a Dutch magazine the recently stated “Developing an app? Rather build a good mobile site”. Research shows that Google could be lucky and the mobile web is likely to win the battle.

Flash and Silverlight applications

The development of HTML5 and CSS3 is a solution to another indexation problem for web search engines. Flash and Silverlight are web applications to make heavy graphic and interactive elements. They can be used within HTML documents and therefore be integrated in websites. The indexation of Flash and recently Silverlight has always been a problem for Google. With HTML5 and CSS3 there is more support on the way for graphic possibilities within your browser. The major benefit for Google is that it easily crawls and indexes this information.

Pay walls

Recently some publishers started working with pay walls for their online newspapers. The major contributor to this development is Rupert Murdoch. He already realized a pay wall for the sites of the Times and the Sunday Times. A Pay Wall basically would also mean that search engines wouldn’t be able to crawl this information. The ideas to introduce pay walls were known for a much longer time for parties like Google. That’s why they introduced the First Click Free program. This program allows users from search engines to find information behind pay walls and access this information for free the first time. For Google it guarantees access to information behind pay walls.

Closed communities

Communities are becoming increasingly important on the internet nowadays. With the major growth of social media communities are easily formed. Some communities are open others are closed, some are web based others are application based. Facebook even was a closed platform for search engines till 2006. Communities within social networks have privacy settings to be able to close them from the wide world. When closed they can’t be crawled.

Other communities are based on a specific platform or application. Take for example Ping, Apple’s social network focused on music. This platform is only accessible through the iTunes apps for iPhone, iPod etc., which aren’t crawlable for web search engines. It’s unlikely Apple will open up this platform for Google’s search engine. Rumors are Google is developing their own Music platform called Google Music which of course will be web based and therefore crawlable.

Other factors

Other factors that could obstruct the crawling and indexation of the information on the internet include:

More web or less web?

As you can see there are many factors threatening the information accessibility on the internet. Google seems to be anticipating greatly on this change and has converted it into a challenge more than a threat. While the web seems in decline the major question for the coming years will be: will we be using more web or less web? The battle between mobile web and mobile apps has just started. Mobile is the future, Google needs it to be open to retain their power on the world wide web. Alternatives for uncrawlable applications in Flash and Silverlight are in development. HTML5 is an important factor in these battles. Furthermore Google is trying to steer the web browser into the direction of an operating system with Chrome OS. Google is working hard to keep the web the center of the internet and to keep the web open and get entrances to the current and coming walls it contains. But will they succeed?

About Jeroen van Eck

Jeroen van Eck is a consultant search engine marketing at the online marketing company E-Focus in the Netherlands.

8 thoughts on “Could information accessibility be the biggest threat for Google?

  1. Great post Jeroen. While I essentially agree with you, there are some caveats:

    – The Wired graph of data usage is highly misleading, as it shows web traffic as % of total internet traffic. If you look at web traffic on its own you’ll find that it is actually growing exponentially (just not as fast as some other internet traffic types, which makes sense as web traffic is generally about much smaller file sizes than p2p and video).

    – Personally I think mobile websites will lose to mobile apps. There’s stuff you can do with apps that you can’t (easily) do with mobile sites, which is why I think apps will eventually carry the day, as I outlined in my post here:

    – HTML5 is still very far from being an accepted and widely used standard. It remains to be seen if it’ll ever get the proper focus it deserves.

  2. @Barry It’s clear that Google hasn’t won the war by far. But they are fighting the battles with the tools they can get. And I think they’re on the right path. Considering your points:

    – I don’t think it’s misleading. The larger the (relative) part Google can’t control the weaker their position will become.

    – I can’t say It’s already clear which one of them will be winning. They both have their own advantages. However the power of the mobile web is the scalability. There is a limit to the amount of apps you will install. I think when the mobile web can compete with apps considering the capabilities, the mobile web will eventually win on scalability.

    – True, but the solution is there. The advantages of HTML5 are obvious. I’m guessing this is just a matter of time especially considering the need of some functionality for mobile web development.

  3. @Jeroen: aye I agree that Google is keenly aware of this threat and is working hard to mitigate it. They’ve moved in to the mobile landscape with Android and its app market, they control most of the video via YouTube, and they’re pushing hard for HTML5 to be the de-facto standard for next gen web apps. Whatever happens, the next couple of years are going to be quite interesting. 🙂

  4. Making it as top search engine is different from staying on top, and it’s clear that Google is trying to do the latter. After dealing with spam, here comes their next challenge.

  5. Google is getting too big, and that will definitely go against their “don’t be evil” slogan, just as with all big corporations. It’s very hard to see a competitive threat to Google. They’re doing everything they can to invest in to what they (and many others) believe to be the next big thing – which is cloud computing. Just look at their browser Chrome and the mirage of web applications they’re developing. They believe computing will move towards a web-based system where all that people will need are a browser and an internet connection. Until Facebook figures out how to mitigate the people’s need for search services, there will be no challenge.

  6. most of the obstacles you mentioned go against the interest of these technology users and developers than Google’s. Companies behind mobile apps and publishers of paid content should work with search engines if they want to attract global clientele.

  7. Howdy, Great article! Very well worded and straight to the point. I am referencing some of this information in my own papers as a guide, I hope you don’t mind.

Comments are closed.