How online shopping has changed quality of service in the travel industry

One phenomenon you see on a cruise is the fact that the cruise company is highly adept at extracting additional revenue from passengers.  It’s a huge exercise in upselling.  You pay extra for drinks and shore excursions and room service and photographs and a myriad other things.  You even pay extra for soda and water.  While cruises and guided tours are the most advanced in this practice, you see the same basic idea applied by airlines, hotels, and rental cars as well.  An obvious example is the surcharge for checked baggage on the airlines.

Here on the cruise I heard a number of other passengers say, “I wish they had just charged me a higher price and then didn’t make me pay for my sodas,” or similar sentiments.  I’ve similarly heard a lot of people moan about the checked baggage charge that comes with air travel.

So if these practices are making customers unhappy, why does the travel industry continue them?  Wouldn’t someone identify this opportunity to differentiate from the competition and include all the extras for free?

No, clearly not.  But why would that be?  The answer is internet sales.

Online travel sales have created an environment where the primary decision point for choosing a flight or vacation or rental car is price.  You go to Expedia (or whatever) and choose the cheapest option that meets your criteria.  Even sites that try to account for other factors (like TravelZoo) still put huge emphasis on price.  And when we all shop for our trips or vacations, we select the lowest price.  Those sites don’t say that this flight comes with free baggage checking - or a meal or a blanket and pillow – while that flight does not.  But it does say that this flight is ten dollars cheaper than that flight.

So with nothing else to go on, you’ll choose the cheaper flight.  Even if it’s a dollar cheaper.  There’s a little more information that goes into choosing a cruise, but the same basic pressure applies.

Now, if you’re in charge of pricing for one of these services, you have no choice but to squeeze your basic, published price down as low as possible.  You have to win the price comparison in this environment.  One of the ways to do that is to carve out everything you can and make it an extra.  Charge for meals instead of including them.  Charge for drinks.  Charge for baggage.  Etc.  That way you win on Expedia, get the sale, and when the customers show up and realize they have to pay to checks their bags, they’re already locked in.

If you happened to have an airline that didn’t sell on these sites, you could differentiate by not charging for these things and promoting that fact in your marketing.  That’s what Southwest does.

So you see, we’re doing it to ourselves.  We all could change this behavior easily by not always choosing the lowest price, or by not shopping on web sites for our travel.  But somehow I don’t think that’s going to happen.

One advantage of James Bond Management

My wife and I are in the midst of a long cruise, and I’m reminded of the last extended vacation we took.  I work in an industry where people tend to drive pretty hard, and I certainly fit that mold.  A few years ago we took a two-week vacation to Greece, and when a crisis communications task came up, I found myself sitting beside the pool in Crete on a conference call to deal with it.  Even though in principle I was supposed to be off the clock and out of touch.

This trip is different in that I’m able truly to put aside those other concerns.  Since I’ve left VeriSign/Symantec but haven’t landed at my next company, it’s been fairly easy to let people know that I’m going to have a few weeks of down time.  They’ll wait.

Comparing the two trips, it’s interesting to notice that I’m not significantly more relaxed this trip than I was on that other one.  Yes, I had to keep an eye on e-mail.  And yes, something came up that forced me to break vacation mode and deal with it.  And yes, the people at the poolside bar gave me a hard time when the call was over for doing work while I was supposed to be getting away from it all.

But in reality is was no great hardship, and the reason is James Bond Management.  I had a strong set of James Bond employees in place at VeriSign at the time and had James Bond relationships with most of my peers in the cross-functional team.  As a result I felt I could have faith in the fact that they would be independent, would exercise good judgment, and could execute on their objectives.  That means the call in question was an efficient, high level conversation.  I was given the facts and could rely on their accuracy.  We discussed options, with ideas coming from all sides.  We mutually agreed to an execution strategy and assigned ownership.  I had a high degree of confidence that the individuals who signed up for tasks would deliver on them or would let me know if they couldn’t.  Then we signed off.

How wonderful is that?  I compare that to managers I’ve seen (or had) in the past who can’t trust and can’t delegate.  One of those managers in that situation would have been stuck on multiple calls and would have been logging into e-mail and dealing with lots of issues to manage an unforeseen business challenge.  I, on the other hand, got to enjoy my time off and still have an excellent resolution to the issue.  And I could really relax, knowing that James Bond was there to take care of me.  I don’t see how a manipulator of what I call hand puppets could feel the same way.

Causation, correlation, and World of Warcraft, part 3 of 3

My previous post describes a double-experience mechanism in World of Warcraft and the fact that it appears to be a deliberate product decision by Blizzard Entertainment to increase the average number of characters per active subscriber.  This decisions feels like the kind of thing a product manager or strategic manager would do based on a statistically relevant correlation between number of active characters and other preferred behaviors, such as propensity to continue subscribing to the service.  See my previous post for caveats around these guesses, but for our purposes today let's assume that all of the above is correct and indeed that the preferred behavior is increased likelihood of continued subscription.

That would make perfect sense.  A big business like this one, which appears to be in excess of one billion dollars per year, is exactly the kind of thing in which increased retention is worth a whole lot of money to the company and for which product initiatives would definitely be on order.  After all, each tenth of a percent of increased retention means more than a million dollars a year to the company.  That's worth spending a little time on analysis.

But here's where we run into the problem with causation and correlation.  Just because there exists a correlation between number of active characters and retention, that doesn't mean that gaining characters is the direct cause of increased retention.  It may be that as time progresses players tend to pick up characters and also tend to increase their overall retention rate.  It's a reasonable scenario that a veteran WOW player might tend to have more characters going and also have a much higher likelihood of continuing the service than a newby would.  But in that case both of the observed metrics owe themselves to a different cause:  Number of years as a player of the game.  In this case, there is a correlation but no causation between the two metrics.  Convincing a newby to have more characters does not necessarily increase retention.

Or maybe the causal connection goes the other way.  Perhaps the super-satisfied WOW player can be identified in part by having a large number of active characters.  Obviously these players would have a very high retention rate.  But again, artificially convincing non-super-satisfied players to add more characters doesn't necessarily address this need.  In either of these scenarios, spending product roadmap time to build out these features and changing the game environment away from a more equitable, less arbitrary state would be wasted, as it wouldn't really address the need the designers have for the game.

It's interesting to notice when these errors occur that we always attribute the characteristic we don't care about (number of characters) as a cause for the characteristic we care very deeply about (retention).  You would never look at these two factors and conclude that driving up renewals is a good method for increasing the average number of characters per player.  Because after all, shareholders don't care about number of characters per player.

Now, all we have to do is look at the stats around World of Warcraft to see that the Blizzard folks are doing a lot of things right, so even if the above speculation is 100% true I wouldn't weep too much for them.  But this basic lesson is applicable to any business with a large installed base or a large run rate or lots of traffic or any other situation that will lead to the temptation to engage in data mining.  And most businesses aren't sitting on the money-printing machine that is WOW.  So in general we would be well advised to try to make these decisions as effectively as we can.

In conclusion, am I saying not to engage in data mining?  Absolutely not.  I do it all of the time.  But I am saying to be careful about drawing conclusions that aren't supported by the data.  In particular I frequently watch people mistake correlation for causation, and that can send you down a path where you're spending your resources and giving your customers a convoluted experience without gaining the benefits you think you are.

Causation, correlation, and World of Warcraft, part 2 of 3

This post examines the phenomenon of data mining as applied to a real world example that anyone can observe.

Let me start by stating that I have seen the practice of data mining and the obscurity of causation versus correlation (all of which is explained in the previous post on this series) occur over and over again in real businesses in which I've worked.  I don't want to discuss the inner workings of those businesses since I had a privileged view on them, so instead I'll pull an example from a popular online game that anyone can see, World of Warcraft (WOW).

A couple of caveats first:  I am not and have never been an insider at Blizzard Entertainment or for the World of Warcraft products.  I know no such insiders.  I also am not a WOW player, although I know people who are.  I'm building this example on something I (or anyone) can observe and my own hypothesis about why Blizzard designed its product that way.  I may be flat-out wrong.  However, that's not important.  Even if this hypothesis is completely incorrect, we can still use it to illustrate how data mining affects business decisions and the potential fallacy therein.

That said, let's start with observable facts.  World of Warcraft is what we call a leveler, a game in which one has a character (or multiple characters - an important point here) in which one can achieve goals an increase the character's powers or abilities, which in general terms can be called leveling up.  Part of the way one levels up in WOW is by gaining points of experience, which are rewarded for defeating enemies or accomplishing other goals.  Now, there are various mechanics in the game by which characters can gain experience faster than they otherwise would to accomplish the same tasks.  For example, there's a mechanic whereby a player can meet up with random other players and mutually go after tasks ("run dungeons") together, and the players will gain more experience for the tasks than if they had accomplished them otherwise.

My first hypothesis is that the folks at Blizzard have created these rich experience earning environments on purpose to encourage certain behaviors in players, behaviors they deem good for the overall WOW ecosystem or simply for their own bottom line.  (Again, let me emphasize that these are my own speculations and that proving them correct or incorrect is not important to the lessons we can learn from this exercise.)  In the above example, I imagine that it's good for the WOW community to encourage players to meet new people and this this mechanism causes that to happen.

That's an easy one.  But let's look at another rich experience earning mechanism that might be a little less obvious.  One phenomenon in WOW is that if you leave a character dormant, the ability to earn experience actually increases.  How it works is that if your character goes unused for some period of time (what they call resting in the game), a limited opportunity will occur to earn experience at double the ordinary rate.  The longer a character rests, the larger the double-earning opportunity, up to a maximum number of experience points.

So we have a mechanism - deliberately built into the product - that encourages players to use characters they have not used recently.  Now, why would Blizzard do that?  A few ideas occur to me.

It's possible that Blizzard Entertainment wants you to do other things besides play World of Warcraft.  The company wants you to go outside and enjoy the fresh air, maybe take in a movie or spend some time with friends.  The company wants to ensure that you keep your WOW consumption to a minimum so that you live a healthy, balanced life, and this mechanism encourages that.

It's possible, I suppose, but that's not the way companies typically think.  Let's look for a more likely explanation.

My more likely explanation is that Blizzard wants to encourage players to maintain multiple, active characters rather than just focusing on one or two.  From what I've seen, WOWers tend to have lots and lots of characters.  An account may maintain up to ten characters, and people seem to push that limit.  Now, under those circumstances there are a few different ways to play the game.  One is to focus in on one or a few favorite characters and play them a lot, touching the others infrequently or not at all.  The mechanism described above encourages players to increase the number of active characters they're engaged with on a regular basis.  The longer a character sits unused, the greater the incentive to pick it up and play it again.  So Blizzard is using the carrot of increased experience points to increase the average number of active characters each player has.

That feels like data mining to me.  My guess is that the folks at Blizzard have looked into the statistics behind their user base in pretty fine detail.  After all, it's a big business with twelve million users logging more than one billion play hours a year.  I'm sure they've analyzed their user base to the nth degree.  I further guess that someone at Blizzard has determined that it's good for them if the average number of active characters per player goes up.  Maybe those players with more active characters renew their subscriptions at a greater rate.  Maybe players with more active characters are also more active in recruiting new players to the game.  Maybe players with more active characters are also more active in other parts of the WOW world and help create a richer experience for everyone else.  Maybe it's all of the above.

Either way, if Blizzard really did mine its customer data and find these correlations, it appears that the product team then chose to encourage players to have a large number of active characters on the assumption that increasing this overall average would improve these other metrics, the ones they really care about.  It could be that these mechanisms indeed have had that effect.

It could be.

But not necessarily.  Join me next time for part 3, in which we'll scrutinize this reasoning in light of the difference between causation and correlation.

Causation, correlation, and World of Warcraft, part 1 of 3

As you know, I love fact-based marketing.  One of the main ways marketers gather facts is by looking at the statistics of customers' behaviors.  Oftentimes that works great.  A/B split testing is a perfect example of statistics-based fact gathering that works really well.  You can determine which price point makes you more money (to use my earlier example) and count that knowledge as highly reliable, even if you don't understand why it's the case.

Another fact gathering method is what they call data mining.  Essentially that means taking an established business, community, or crowd of any sort and looking for patterns and trends in its behavior.  Marketers often data mine by looking at the customer base or transaction records for a certain period of time and trying to discern markers or behaviors that directly correlate to the behaviors they ultimately want.  For example, customers with this one set of characteristics are more likely to buy an upgrade than customers with this different set of characteristics.  Marketing and sales managers often view these correlative discoveries as gold strikes which will yield great dividends for the business.

Certainly data mining is a key tool in fact-based marketing, but it's laced with danger.  For this series I'm going to focus one of these dangers, the difference between causation and correlation.  I watch lots of business people engage in data mining exercises, and yet I often find that I'm the only one in the room who is alert to the difference between these two qualities, even though the difference is of Earth shattering importance.  Let's start with definitions.

• Causation:  The relationship between a pair of events or circumstances in which one of them is the direct creator (or cause) of the other.
• Correlation:  The relationship between a pair of events or circumstances in which the presence of one affects the likelihood of the presence of the other.  In the case where two circumstances are less likely to occur in tandem than the average, we tend to refer to it as negative correlation.

The difference is subtle but extremely important.  In a causal relationship, event A brings about event B (or makes it more likely).  In a purely correlative relationship, event A does not necessarily bring about event B.  Perhaps event B brings about event A.  Or perhaps both are brought about by a separate event C.


As it happens, causal relations are always correlative relationships.  Causation is a subset of correlation.  It also happens that, unless you believe in magic, correlative relationships imply some kind of causal relationship somewhere.  It just might not be the one we're looking at now.

Example:  In our house we have two dogs, Bruce and Max.  Bruce and Max tend to run around on all fours, while my wife and I tend to walk around on two feet.  Likewise, my wife and I have well developed frontal lobes, while the dogs' foreheads are pretty much full of skull.  If we looked at large numbers of dogs and humans we could see these two patterns consistently occur.  Therefore we can conclude that there exists a correlation - when looking at a population of canine and human adults - between possessing a well developed frontal lobe and walking around on two feet.  However, anyone with even a small knowledge of biology will know that the frontal lobe does not cause the walking on two feet nor that walking on four legs somehow eliminates the frontal lobe.  Instead we know there are other causes of both these syndromes, ultimately going back to a root cause, which in this case is differences in DNA between the two species.

In the above example, it seems pretty obvious.  But I routinely watch business people mine their data and then jump to conclusions without ever asking this question.  Next time, in part 2, a (conjectured) real-world example.