May 27, 2006
How shopping cart recommendations came to be
This is a pretty cool story of how experimentation can quickly show whether something is a good idea, or a bad idea:
Geeking with Greg: Early Amazon: Shopping cart recommendations
I loved the idea of making recommendations based on the items in your Amazon shopping cart. Add a couple things, see what pops up. Add a couple more, see what changes.
The idea of recommending items at checkout is nothing new. Grocery stories put candy and other impulse buys in the checkout lanes. Hardware stores put small tools and gadgets near the register.
…
I hacked up a prototype. On a test site, I modified the Amazon.com shopping cart page to recommend other items you might enjoy adding to your cart. Looked pretty good to me. I started showing it around.
While the reaction was positive, there was some concern. In particular, a marketing senior vice-president was dead set against it. His main objection was that it might distract people away from checking out — it is true that it is much easier and more common to see customers abandon their cart at the register in online retail — and he rallied others to his cause.
At this point, I was told I was forbidden to work on this any further. I was told Amazon was not ready to launch this feature. It should have stopped there.
Instead, I prepared the feature for an online test. I believed in shopping cart recommendations. I wanted to measure the sales impact.
I heard the SVP was angry when he discovered I was pushing out a test. But, even for top executives, it was hard to block a test. Measurement is good. The only good argument against testing would be that the negative impact might be so severe that Amazon couldn’t afford it, a difficult claim to make. The test rolled out.
The results were clear. Not only did it win, but the feature won by such a wide margin that not having it live was costing Amazon a noticeable chunk of change. With new urgency, shopping cart recommendations launched.
When in doubt, try it out. Get some data. Avoid confirmation bias!


