How Amazon works – courtesy of David Gaughran

I don’t often reblog almost an entire post, but I thought this one was too interesting, and informative, to pass up.

Amazon’s Recommendation Engine Trumps The Competition
Posted on February 22, 2013 by davidgaughran

There’s an old adage that bestsellers are chosen rather than made, and there’s some truth to that. The amount a publisher splurges on the advance has to be recouped before the book turns a profit. The more money that has to be recouped, the greater the marketing budget.

Sleeper hits are the exception for a reason. It’s a lot easier to hit the bestseller lists when you are on the front table of every single Barnes & Noble than if you are spine-out at the back of a handful of stores (or gathering dust in the warehouse).

It often comes as a surprise to those outside publishing that these bookstore spots are bought and sold, that whether a book is face-out or spine-out (or on the front table) is something that tends to be agreed in the contract between the publisher and the retailer. But when you explain how valuable this “real estate” is, it all makes sense to them (even if the scales fall from their eyes a little).

It’s very different on Amazon – where a weird form of meritocracy decides which books are visible, rather than backroom deals only available to large publishers. While Amazon hasn’t done away with “virtual co-op” completely, the vast majority of slots where books are recommended to customers are open to any book, author, or publisher – if they perform well enough.

When it comes to books, Amazon’s basic philosophy is simple: it will always (attempt to) show you the book you are most likely to purchase. The system is largely agnostic, meaning Amazon doesn’t care if the book it displays is published by you, me, them, or Penguin, and it also doesn’t care if the book is 99c or $14.99 – it will show you the book you are most likely to purchase.

In simple terms, the system is based on aggregating data about your browsing, purchasing, and reading habits, and then extrapolating about what you would like to read next based on all the other customers with similar histories (we aren’t as unique as we’d like to think).

Those recommendations manifest themselves in different ways. One of the crudest iterations is the Also Boughts (that strip of books on the product page of your book, which displays the other titles that customers have purchased along with your cri de coeur).

At the other end of the scale are the millions of personalized emails that Amazon sends out to its customers every day with tailored purchase recommendations, like this:

Some of you may quibble about how useful those recommendations are, but you should note that writers especially may get some odd suggestions (not the above, which I’ve heard is great!). You have to remember that your browsing habits play a big part in this, and if you are regularly stalking other books to check on their performance, Amazon will likely recommend these to you (along with similar books).

For customers with “purer” browsing histories though, the recommendation engine can be spookily accurate (and is widely considered to be the best in the e-commerce world). And, of course, its accuracy increases every time you browse, purchase, and read, and with every huge chunk of investment Amazon makes in honing its algorithms.

It’s quite the challenge for physical bookstores. In a recent post, Passive Guy described it perfectly:

When a customer walks into a Barnes & Noble store, is it possible for a clerk to be waiting at the door with a selection of books that the customer will probably want to read? This is exactly what happens whenever an Amazon book purchaser visits the Amazon web site. As a matter of fact, Amazon performs the electronic equivalent of rearranging a Barnes & Noble so all the visitor’s favorite book types are right at the front of the store.

Of course, Barnes & Noble also has an online store, and Amazon faces additional players in the e-book arena like Apple and Kobo. But all of Amazon’s online competitors share the same fundamental flaw: the customer experience is considerably poorer.

Kobo’s search function is deeply flawed. Apple make it plain difficult to simply browse. And the problems with Barnes & Noble’s online store are so widespread, I needed a whole separate post to spell them out.

While some of the disparity in customer experience is down to lack of investment, simply throwing money at the problem won’t close the gap because there are two distinct philosophies at work.

In the case of all three of Amazon’s primary competitors, it’s quite clear that they want to train customer attention on that “virtual co-op” – the prominent spots that large publishers have purchased to hawk their books.

I’m sure these retailers make good money from auctioning off these spots, and I’m sure they are also quite pleased that the books they are granting this all-important visibility to are ones priced at $9.99 rather than 99c.

But it’s a huge mistake. Explaining why will require a little detour to Silicon Valley.

The reason that Google beat Yahoo is simple: relevance. While Yahoo auctioned off advertising spots to the highest bidder, Google’s AdWords made the relevancy of the ad (decided by the click-thru rate) a key component in deciding which ads got the prime real estate above search results.

Google knew that approach might make them less money in the short term, but it also knew that, over time, users would trust the ads more (i.e. click on them more), if they were more relevant. And we all know what happened next.

I’m sure Amazon was watching that battle as their recommendation engine takes the same approach: it always shows readers the books they are most likely to purchase even if that recommendation makes them less money than the alternative.

Amazon knows that if its customers trust the recommendations, they will act on them more often (and spend more money). They know that will make more in the long run.

That’s why Amazon is winning. But it’s also why self-publishers tend to do much better on Amazon then elsewhere – even when you factor in the size of its market share.

We don’t tend to have access to the front tables at Barnes & Noble – either online or in the physical stores. The whole point of those front tables is to draw readers’ attention, to intercept them before they start browsing the shelves.

Amazon gives us much more of a level playing field; those all-important opportunities for visibility – those digital front tables – are open to everyone. And because those front tables are displaying the books customers actually want to read, rather than the ones large publishers most want to sell, people buy more books.

The danger for large publishers is clear: they aren’t just losing control of which books get published, but also which books get recommended.

You can find the complete post, which includes some other interesting, although non-related tit-bits here.



About acflory

I am the kind of person who always has to know why things are the way they are so my interests range from genetics and biology to politics and what makes people tick. For fun I play online mmorpgs, read, listen to a music, dance when I get the chance and landscape my rather large block. Work is writing. When a story I am working on is going well I'm on cloud nine. On bad days I go out and dig big holes... View all posts by acflory

12 responses to “How Amazon works – courtesy of David Gaughran

  • metan

    I have noticed recently that Amazon have picked a few great things in their recommendations for me, the little conspiracy theorist in my head was shouting loudly at me even as I bought them!

    I hate giving in to advertising but sometimes they actually get it right…


    • acflory

      lol – I know, it starts to get a little sci-fi when you think a computer has done the choosing. 😀


      • metan

        Damn that Amazon for causing the housework to build up.

        I have spent the last week with my head buried in the Rho Agenda trilogy thanks to those bloody recommendations.

        Books like that (and the online advice that led me to them) aren’t a balm to my suspicions of the ubiquitous ‘them’ knowing everything we do….. *insert twighlight zone music* 😉


        • acflory

          Oh? Rho Agenda? Sounds interesting. Any good? I’m going to have to restock my Kindle very soon.


          • metan

            Excellent. Alien craft, government cover-ups, technology that takes a better brain than mine to understand but explained interestingly enough to keep me reading nevertheless….

            A few times I have shouted “No! Look out/Don’t go there/do that!” at my kindle too. Always the sign of a good story to me.

            I haven’t finished it yet so there had better be a satisfactory ending otherwise you will be hearing my scream of anguish from where you are 😉


          • acflory

            lol – I hope I don’t hear anything coz this really does sound interesting!


  • josh

    This is interesting. I think Amazon is more successful with some type (like the mentioned “pure browsers”) than others. I haven’t purchased books from Amazon for quite some time. I have a Nook reader so I buy ebooks from B&N. I used to be in a couple books clubs and I’d occasionally buy from Amazon if I couldn’t get an ebook or a physical book from the library. Now I tend to mostly buy ebooks, though I do still sometimes search Amazon as they tend to have a higher volume of reviews.

    I do shop Amazon for other things, and the recommendation engines works for those as well. But I get a fair number of emails from Amazon and I generally ignore them. I’m not much an online impulse shopper.

    I didn’t realize that the prominent positions in brick & mortar bookstores were bought & paid for, but I can’t say I’m the least bit surprised. Given the ebook price fixing I’ve seen, I find the industry rather sleazy. Just like every other industry. 😉


    • acflory

      Yeah, the sale of ‘position’ surprised me too, and sleazy pretty much covers it.

      These are the times when I’m reminded how little most consumers know about what goes on behind the scenes. At least Amazon cuts us some slack. 🙂


      • josh

        I just read an article (excerpted from an upcoming book I believe) about how food companies make & market their products–especially junk food–and “sleazy” applies there too. One of their successes was upgrading potato chips from an occasional snack to a valid part of a meal.


        • acflory

          Gah… don’t get me started on junk food. All over the Western world we boo hoo about the obesity epidemic without acknowledging that junk food is to blame. Now food manufacturers are making even more money off the problem they created by marketing sugar-free, fat-free etc. But the sugar substitutes are worse than sugar and fat-free substitutes sugar and salt for fat. Endless, vicious circle while the cash registers keep going ka-ching. 😦


  • Honie Briggs

    This is interesting for many reasons, but one that is relevant today is that earlier this week I posted the outline for a garden talk that I delivered with links for gardening books that I own. Of course I tested each of the links before publishing my post. Today there is nothing but gardening books recommended for me on Amazon.Hilarious!


Don't be shy!

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: