Speaker 1: Humans have always had to make important decisions, often in groups, and for most of history they've used the same basic technique, whether they were deciding whether to have their village go to war with a neighboring village or whether to introduce a new product into the marketplace. What they would do is get the group of the wise people together sitting around a table and everyone would explain why they thought one decision or the other should be made, and after some period of debate and perhaps even a little bit of evidence they would go with the HIPPO. Well, what's the HIPPO? The HIPPO is an acronym that our friend Ron Cavahavy at Microsoft came up with and it stands for the Highest Paid Person's Opinion. That's right. After all the debate, one person, the chief, would make the decision based on his or her own decision-making instincts. That worked tolerably well for lots of decisions, but now we're moving from HIPPO-based decision-making to data-driven decision-making, and a great example is what Jeff Wilkie taught me. Jeff is the head of Worldwide Consumer at Amazon. He's also a Sloan grad, so every once in a while he comes back and gives me a few tips. I remember some time ago he came to my office and we were talking about data-driven decision-making and the way things were evolving at Amazon, and before we really got into it he said, hey, let me bring up Amazon. I want to take a look at your website, and I said, okay, fine. So I brought up Amazon on my web browser and he looked at it for a second and he says, ah, you're in group B. I said, what do you mean I'm in group B? He goes, well, we're running an experiment right now on you and millions of other people. Group A has the shopping cart on the left and group B has the shopping cart on the right. Sure enough, I looked at the screen and there was a little Amazon shopping cart and it was over on the right side of my screen. I said, well, what difference does that make? And he said, well, at Amazon there was a debate whether or not it made more sense to have the shopping cart on the left or the right, and we had some experts telling us, oh, for psychological reasons it makes more sense to have it on the left or the right, but we just shut down that debate very quickly and we said, let's run an experiment. And so that's what they did right then and there. I found out a couple weeks later that it turned out that having the shopping cart on the right was worth a few extra tenths of a percent in terms of getting people to not abandon the shopping cart. Who could have guessed that? Probably no expert could have, but by doing an experiment and gathering data from millions of customers, they were able to find out that it's slightly more likely that people consummate their purchase if they have the shopping cart on one side or the other. These kind of A-B tests are ubiquitous, not just at Amazon, but at every major digital company, and now companies across the world are using this experimental approach to gather data. And once they have enough data, they can make a fact-based decision instead of one based on just guesses and intuition.