The Heart of Artificial Intelligence

The Heart of Artificial Intelligence

17th August 2017

My son Arthur has just been awarded a prize for story-telling at his primary school. So when I watched that short movie whose script was generated by artificial intelligence, based on thousands of sci-fi books and films, I could certainly see a lot of similarity between both outputs. For me this epitomizes the current state of AI… It is raw, forming, full of potential but still with a long way to go towards maturity.

Today we will be covering the heart in artificial intelligence. After all, if artificial intelligence is, by definition, artificial, how can it have a heart, how can it have emotions? On the other hand, if AI is the brain child of thinking, feeling people, how can it not have a heart? This is a critical question for us, marketers who want to trigger emotional reactions from consumers but who also rely always more on algorithms and automation. To answer it, we will first need to define artificial intelligence. We’ll explore the 7 outcomes we can expect from AI, and consider how we materialize these expectations today to enable everyone of us to fulfil our potential.

The building blocks of AI

To understand if a heart is beating inside artificial intelligence, we need to understand where AI comes from. Although AI has been all over the press lately, it is not news… It is rather a 30-year old corpus of work, aimed at creating intelligent machines, by combining three building blocks: machine learning, human learning and data science. And in many ways there is a strong analogy between AI and raising a child.

Cedric Chambaz: the building blocks of AI

Just like children get their foundational learnings from their parents, teachers and by the school books they read, machine learning is based on known properties, and the machine learns from the data. Think if/then scenarios. If your son behaves well, then he will be treated by Santa. If your daughter sees a puddle, then she should not to stomp in it to keep her feet dry. This is also how machine learning works: if you liked that book, then you’ll probably like these ones too. If you bought a laptop, then you should consider this bag. These are just small, basic examples of a very complex field.

Kids learn fast that if they cry and shout they get your attention… Now, you will certainly want them to assimilate that such a behaviour is not a normal mode of expression. Human learning is how we make course corrections to the machine learning that is happening. Cortana, Microsoft’s digital personal assistant, has a team behind the scenes working on human learning so she can get smarter. This human learning gives the digital personal assistant more personality, and her responses to queries are more human because of this.

Data science is the third brick of artificial intelligence. Data science is the discovery of unknown properties, or connections, in data. In this case, the machine is presented with a massive amount of data and asked to find connections in it. This is how we might discover that watching a certain program in your youth increases your chance to marry a foreigner. We didn’t know there was a connection between these pieces of data until we went looking for that connection.

Human intervention

We have seen these three fields accelerating their capabilities recently due to the exponential development rate of our computing power. We are able to process, analyse and render an ever-growing amount of information, at an increasing pace. But where does that data that feeds machine learning, human learning and data science come from?

It comes from us! Artificial intelligence comes from us. In many ways, it is us.

Artificial intelligence is only as intelligent as the data it takes in. It is only as fair as the data it takes in. It is only as human as the data it takes in. It is only as socially acceptable as the data it takes in. I would like to share with you two examples of AI, which to a large extent illustrate how humans can influence how intelligent a bot can be.

Remember Tay, Microsoft first experiment as a Twitter bot? Tay learned from her inputs, which were hijacked by some people who wanted to influence her negatively. In this case, Tay incited high emotion from people who engaged with her or read about what happened with her, even if Tay, herself, did not express emotion and was merely a reflection of the hatred that fed her.

On the other end, Microsoft also created Xiaoice a couple years ago and it is a perfect example of where technology is going and why we think of conversations as a new platform for brands and commerce. Xiaoice is a chat-bot based on Bing search technology and big data. It draws on AI, social media, and machine learning so she can hold a proper conversation – the average exchange between Xiaoice and a user has 26 turns. She’s sensitive to emotions and remembers your previous chats. If you tell her about a breakup, she’ll check in with you. If you introduce her to a puppy through a photo, she’ll recognize the breed, ask you for its development. And to say this bot has been popular is an understatement. Three days after she was available, Xiaoice had been added to 1.5 million conversations on WeChat. Once added to Weibo, the Chinese micro-blogging service, it became one of the most popular celebrity accounts. And today, Xiaoice is used by over 40 million people.

Tay and Xiaoice are like two twins, split at birth and raised in two different environments, with different influences… Two very different individuals in the end.

Assessing our expectations

So, what can we reasonably expect from artificial intelligence?

As mentioned before the computing advancements have enabled a fast acceleration of three technologies which underpin the maturation of artificial intelligence: object recognition, natural language processing and speech. If the AI can see, speak and listen, it is not far from being able to exchange with human being transparently.

Actually, mid-October 2016, Microsoft researchers announced they had reached human parity with the word error rate (WER) for conversational speech recognition, meaning that their AI was as capable as a professional transcriber to write up an oral conversation. Language understanding and acquisition is not easy, and it is critical to the success of AI. If you travelled a bit, you will be familiar on the complexity implied by accents, dialects, pronunciation but also the fact that a same word may have several meanings based on the context. This progress was critical because without this piece of artificial intelligence, so many developments wouldn’t move forward. Think about how patient you would be with a digital personal assistant or a sales advisor that misunderstood most of what you said?

Natural language learning is a complex skill, as we know from watching our children learn to speak. But with our increased computing capabilities, not only are we able to recognize accurately the words but we are able to do this instantaneously. This unlocks new scenarios like Voice-to-text which allows deaf children to read the transcript of a discussion in real time or Skype Translator which not only has the natural language skills necessary for a conversation but can also translate into other languages.

Well, this outcome is one of many. Capitalizing on the progress of machine learning around object recognition, natural language processing and speech, we have seen our expectations towards AI graduate from the most basic to much more advanced outcomes.

The 7 outcomes of AI

According to Silicon-Valley analyst, Ray Wang, there are seven intertwined outcomes for artificial intelligence, based on what we are now able to program via machine learning.

Cedric Chambaz: 7 outcome of AI

  1. Perception is an example of early machine learning, now totally engrained in our daily life. Drawing on existing data, the machine delivers information about what is happening now. The weather, traffic, sales volumes, stock prices – things that are measureable and reportable. This AI outcome brings us back to the core promise of search engines when based on a typed or voiced query, the machine learning understands the intent and provides the answer or links to the information. For humans, learning to express their perception, it’s pretty simple as well. A child can describe what is happening now with ease. We learn this almost immediately: it is dark; I am hot; or, based on these circumstances, I am joyful. To illustrate a more advanced Perception outcome, we can look at facial recognition and play with which assesses your age based on your traits (and which we hate to be accurate).
  2. Next, Notification. If I did not have my calendar delivering notifications, I would be a horrible colleague – late to meetings or just not showing up because I cannot hold my schedule in my mind. Here the intent is less explicitly verbalized, but it is still initiated by the user and the information remains factual without any analysis of the data. We learn notification early as well, perhaps starting with letting Mom know we’re hungry. Fact: I am hungry; Notification: I cry. It never stops – in school, we notify the teacher that we have the answer.
  3. Suggestion is another area we have grown to be familiar with, and is now engrained in our daily life. You searched for these words, but “Did you mean?”… The machine learns from past behaviours and suggests alternative actions. We all love this machine learning with our Spotify account for instance. If I listen to a song and I like it, the AI suggests more songs for me to enjoy. And you can always retain that Human Learning capability to ensure that the AI never drifts from Justin Timberlake to Justin Bieber… Early suggestions were basic, but imagine what can influence them today: demographics, location, day, time, weather, behaviours, etc. The data sets are humongous but we are now capable to combine and process them in no time and identify new, maybe more obscure connections
  4. Our children learn a nice drawing will trigger a smile from their parents, or that it’s time to wash their hands before a meal. Over time, we don’t even have to remind them; they just know it’s what’s next and it becomes Automation. A suggestion or a recommended action can grow into automation based on learning your preferences. If you follow avidly the progress of your favorite team, the AI will start to automatically inform you of their performance. If you always make a reservation for 7pm on Saturdays, your AI will start to spontaneuously fill in the date and time on your reservations. If you trigger the same report every Monday morning, the machine will start to pull the information for you and make it available in your Business Intelligence dashboard.
  5. Predictions can be the hardest machine learning to train, because so many variables can affect this outcome. Think of a child who sees Daddy packing a suitcase; based on past behaviour, this toddler knows that this means Daddy is leaving for a few days, which is sad. But sometimes it also means that the child gets to travel with Daddy. What factors will alert the toddler about what outcome to expect? Microsoft has developed a program called Bing Predicts which combines and models all the data signals we can find, and comes up with incredibly accurate predictions. It initially explored popularity-based contests like American Idol, for which the web and social signals are very strong and highly correlate with popularity voting patterns. You search for information about that performer, his history, his latest video clip. At the same time, you comment the performance on Facebook or Twitter. By combining anonymized search patterns to social signals Bing Predicts could accurately project who would be eliminated each week during American Idol and who the eventual winner would be. More complex, we then turned to sporting events and even world political challenges. During the World Cup in Brazil, our team predicted accurately with 100% accuracy the winners of the final elimination round. During the last year Rugby World cup, we had 87% accuracy across the tournament. Surprised? In order to successfully predict a sporting event outcome, the number and type of signals we incorporated quadrupled from what we used to predict a basic popularity event like American Idol. This is because we recognize that popularity alone does not predict whether a team will win – Sorry for the fans. A fan base has however special insight into the abilities of their teams, and those fans are having constant discussions about their team. This is called the “Insider Knowledge.” We weighted their knowledge against player and team stats, tournament trends, game history, location and even weather conditions. This is how we were successful in our predictions.
  6. If we manage to predict accurately the future, the next logical step after prediction is Prevention. Again Bing Predicts shines in this category: by analysing large samples of search queries, Microsoft scientists have been able to identify internet users who are suffering from pancreatic cancer even before they were diagnosed. The researchers focused on searches conducted on Bing that indicated someone had been diagnosed with pancreatic cancer. From there, they worked backward, looking for earlier queries that could have shown that the Bing user was experiencing symptoms before the diagnosis. Those early searches, they believe, can be warning flags.
  7. Finally, Situational Awareness for AI comes close to mimicking human behaviour in decision making. We see situational awareness as a combination of many aspects of AI, from object recognition to conversational speech. Here’s an example:

These 7 outcomes are complex and require a lot of training and time to accomplish. They are also interconnected and not mutually exclusive. They actually build upon each other to offer the benefits of AI to us, users.

In conclusion, everything we’re seeing with AI is exciting and rich. We see the heart in AI every day, when we ask it to help uncover cancer, help two people connect when they don’t speak the same language. But where is the moral and ethical compass for artificial intelligence?

As alluded to through this article, AI is still at its infancy and it is our collective responsibility to set it on the right trajectory. At Microsoft we are committed to this, and partnered with the University of Cambridge and the Partnership on AI, two international authorities to help shape the future of that promising discipline. For some, AI is a modern Oedipus that will have to “kill the father”, take away our jobs, make ourselves redundant. But for someone like Satya Nadella, AI will actually enable people to fulfil their full potential as we have seen across the 7 outcomes of AI. So yes, for Microsoft, AI has a heart. It is the mankind’s heart.


Written By
Tasked with promoting Microsoft’s search marketing platform - Bing Ads – around the world, Cedric works closely with advertisers, from Blue Chip brands to small businesses, to help them develop effective online strategies which can boost their visibility with PPC, provide more targeted communication and deliver higher ROI.
  • This field is for validation purposes and should be left unchanged.