Thursday, April 28, 2011

Why I love A/B split testing

I love split tests as a marketing tool.  Love 'em.

First, a quick definition:  Split testing is when you take a large number of potential respondents to an offer and split them up into different groups - or cells we call them - to receive offers or communications that are somehow different from each other.  If the respondents are homogeneous across the groups, then  you can reliably test how the differences in the offer or communication affect the respondents.  A simple example.  I could offer the same product in direct mail to the same list with the same creative, but in one case I could charge $99 while in the other I charge $79.  Now I have tested to see which of these two price points makes me more money.

The above example would be an A/B split test because there are two cells, A and B.  If there were three cells it would be an A/B/C split test.  Etc.  In general, however, the number of cells isn't that important to the concept of the test, and therefore I notice that in the real world A/B split testing and split testing are used almost interchangeably.

In theory you can conduct split tests under all sorts of circumstances.  For example, large retail chains conduct them by sending different in-store promotions out to half their stores.  Once upon a time the bulk of split testing took place in conjunction with direct mail.  Today there's still a lot of split testing in direct mail and direct e-mail, but the most interesting work is happening on web pages.

That's because web pages are the dream environment for this kind of test.  Homogenizing the traffic is a piece of cake.  You can get results in real-time and change the tests as often as you like.  There are many robust tools out there to help with your split testing, and they're super easy to implement and use.

And of course because the rewards are so high.  Split testing is the single best tool for taking you out of speculation and into facts.  The example above is incredibly important in terms of that product's revenue and profit.  You can test all kinds of things - message, purchase flow, offer, product name, headline, you name it.  The depth and variety of market knowledge you can gain is limited only by your tools, your traffic, and your imagination.

Saturday, April 23, 2011

The now vs. not typo

While they're unfortunate, usually typos just have the effect of being a little jarring or distracting.  At worst they make you look sloppy.  However, there is one typo of which I've been aware for many years that is a particularly damaging one.  You want to stay vigilant to keep it from affecting you.

I'm talking about the now vs. not typo.  Quite simply, that's when you mean to type one of these two words but your well-practiced fingers type the other instead.  Obviously this typo would only affect touch typists, but there are plenty of us in the world.

This typo is particularly insidious for many reasons:
  • It's easy to make.  It only requires the substitution of a single letter using the same hand.
  • Spelling checkers don't catch it.
  • Grammar checkers don't catch it.
  • It can occur both ways (now to not and not to now).
  • It has the effect of completely reversing the meaning of the sentence.
This last point deserves a little attention.  Let's say you're a one-man development shop that creates apps for smart phones and sells them on the popular application stores.  You create an iPhone app called Zombiesquare.  It's like Foursquare but with zombies.  Naturally it's a big hit.  Next thing you know you have lots of requests to build a version specific to the iPad, which will take advantage of the larger screen to show the zombies in their full technicolor glory.  You go through the trouble and expense of creating an iPad version.

Now it's time to promote it.  You quickly add some copy right up at the top of the Zombiesquare home page stating,
Zombiesquare is now available for the iPad.
However, what you accidentally post is
Zombiesquare is not available for the iPad.
Wow, that sucks.  You just took your very best set of target buyers and told them not to bother.  That'll definitely cost you some sales.  All because of a stupid typo.

The other direction is just as possible and just as bad.  Think of how dangerous it is if your coupon is supposed to have mouse type that reads, "Not available in combination with any other offer," but instead states, "Now available in combination with any other offer."

So when I'm proofreading my own or others' work, I always stop on the words not and now and take a beat to ensure they mean what they say.  I recommend the same to you.

Wednesday, April 20, 2011

Introducing James Bond management

I've been directly managing product and marketing teams (and indirectly managing cross functional teams of all disciplines) for almost twenty years.  Over that time I've evolved a philosophy of management that I call James Bond management.

To understand James Bond management, let's use the example of - you guessed it - James Bond.  James is a secret agent, which means his job is to go into difficult situations about which the facts are sketchy at best and drive a satisfactory resolution.  The mission goes something like this:  "James, our man in Bermuda has stopped calling in.  We don't know what's up.  Here's your plane ticket and your false passport and a watch that squirts acid.  Go figure it out and call us if you need to."  Now James wings off to Bermuda and starts sniffing around and makes decisions based on what he learns.  Sometimes it turns out that the agent in Bermuda is off drunk somewhere and the problem is easily managed.  Sometimes it turns out that SPECTRE is involved, and then it's a much bigger deal.

But what doesn't happen is the folks at home sitting and micromanaging James's every move.  That's because they can't.  James is halfway around the world dealing with things in real time, and there's no way M can look over his shoulder and say, "Throw the knife now!"  The folks in London instead have employed a highly capable, highly empowered agent, and they give the agent room to do his work.

To the fullest degree that I can, I try to use the same philosophy.  Each employee is a secret agent whose purview equals that employee's job description.  Each is given James-Bond-like authority within that purview.  Just like James these employees enjoy as much latitude as they can handle, and just like James they all must hold themselves accountable to results.

Now, that doesn't mean you just throw them into the wild and say, "Good luck with that."  James Bond receives training and information and tools critical to his success, like the aforementioned watch that squirts acid.  So do James Bond employees.  James also knows where his boundaries are and when he's stepping beyond them.  James can't order an air strike, no matter how important he thinks it is.  But James can request an air strike and explain why it should happen.  Likewise James Bond employees enjoy clarity on what decisions they may make and where they become recommenders and influencers instead.

I mentioned above that James's employers take this attitude because they simply have no choice.  To do otherwise is to guarantee failure.  I contend that the same is true for most information workers.  There is an old adage stating that if two people have the exact same opinion then one of them is unnecessary.  Over the years I have gotten great results by encouraging my James Bond employees to think for themselves.  Oftentimes they've innovated wonderful improvements that I myself had never considered.

Of course, while trust and freedom are critical to successful James Bond management, they only work if your employees really are James Bonds, at least in their specific areas.  That comes about through recruiting, mentorship, and culture.  I plan to get into more detail on how to attract, create, and maximize your James Bonds in future posts.  Stay tuned.

Monday, April 18, 2011

Why I love fact-based marketing

I'm a big advocate of fact-based marketing, which means I like to focus on what we can determine objectively about the market and the customer as much as possible, relying on our own opinions and impressions only when we have no other choice.

This approach gives us many advantages:
  • Accuracy.  First and foremost, the more outside information we can gather, the more likely we are to make optimized decisions.  If you discovered a new island, you wouldn't imagine mapping it by sitting down with a piece of paper and making things up.  You would go look at the actual island and draw what you see.  Yet, people map a market all the time simply by sitting down and making it up.
  • Repeatability.  If the facts gathered the first time were accurate, we can have a high degree of confidence they'll still be accurate in the future.  That means we can apply the lessons we've learned over and over again.
  • Teachability.  The fact-based approach is something marketers and teams can learn.  That means it can go on helping the organization be better even when entirely new individuals are involved in doing the work.
  • Extensibility.  Once we're building a structure of empirical knowledge about a market and its behavior, we can start to make connections.  We can make reasonable conjectures about other behaviors we'd expect so see, even if we don't have the facts on that matter yet.  Just as physicists have used their demonstrated knowledge to reason out good hypotheses to explore new questions in physics, marketers can do the same thing.

Thursday, April 14, 2011

Introducing the external-internal ratio

If you've been reading earlier entries in the Tim Callan on Marketing and Technology blog, you've seen that I tend to invent these frameworks to aid thinking about how and what to do in business.  That's what the baskets-versus-fruit parable is.  That's what the five qualities of successful experimental marketing programs are.  That's what the Seven Habits of Highly Effective Marketing Departments are.  I have lots of them.  Here's a simpler one that's a little quicker to explain, but still very useful in thinking about how you allocate your resources, time, or attention.

The external-internal ratio is a simple method of thinking about how you allocate your focus.  The basic idea is that if you're in any kind of marketing, sales, or customer service role, you want to maximize the percentage of your time and effort that directly influences people outside the company.  Better to spend an hour working on web copy than an hour working on a memo for your VP.  Better to drive out and visit a large account than spend a morning in a cross-functional meeting.  Better to focus on influencing a journalist who covers your beat than a manager in another department.

If you've taken any Pragmatic Marketing courses you're probably familiar with the term NIHITO.
NIHITO:  Nothing important happens in the office.
That's a variation on the same idea.  A bit of an overstatement, perhaps, but it makes the point.

Obviously it wouldn't make sense to be purely external facing.  For one thing, oftentimes we can use internal communications, meetings, and influence directly in the service of better external facing activities.  But with a little common sense the external-internal ratio still applies.  Building a slide deck that teaches your creative agency how to communicate better with your customers is much more external than building a slide deck explaining the business to the CFO, at least in most organizations.  Interacting with the sales team or the customer support team oftentimes counts as a highly external-facing task because these individuals are either passing on information about the market or learning ways of becoming more effective in dealing with the market.

I challenge myself and my team to be as external facing as we conceivably can.  Do you have to have this meeting?  Can we make this meeting shorter?  Do all these people have to be here?  Are you spending a lot of time carefully crafting e-mails when you could just give it to me unadorned?  Are you making fancy presentations when you could just be sharing a few facts?  I also try to walk the walk by being a manager who doesn't require lots of internal management.  I encourage those around me to be the same way.

Monday, April 11, 2011

How I got involved in user experience design

For my entire career as a software and internet product manager and product marketer, I've been a strong proponent for focus on optimizing the user experience.  Here's the story of how it all began.

I majored in English.  My thinking at the time (which turned out to be valid, by the way) was that most people are in jobs that you can't major in, implying that most people are doing something other than their major in college.  So while many of my friends were choosing to major in organic chemistry or electrical engineering, I instead spent four years of my life on something I absolutely loved.

Coming out of college I did have the practical problem of getting a job.  I asked myself, "What salable skills do you have, Tim Callan?"  One obvious answer was writing.  So I hunted around for writing jobs, and eventually I went to work for a small Windows ISV as the documentation department.

My first job was to create all documentation for upcoming product releases.  That meant writing and layout for manuals and quick start guides and creating help systems.  So I had to sit down and explain how to use our powerful but complex products.  Complex and too often counterintuitive.  I found myself writing long explanations to make sense of obscure functions.  I found myself going to great pains to emphasize or highlight a few key product points that would prevent people from getting lost in the weeds of the product's functionality.  I found myself patiently articulating how a certain function actually behaved, as opposed to how you'd expect it to behave.

Let me give you a simple example.  Our flagship product had a calendaring function and one of the things you could do was set alarms that would go off at specific times.  It happens that there were no concepts of noon and midnight in this product.  Rather there were 12:00 am and 12:00 pm.  Now, it wasn't obvious to me which of these meant noon and which meant midnight.  I mentioned this fact to the VP of development, who said, "12:00 am is midnight, of course."  Then I mentioned it to the VP of marketing, who said, "Everybody knows that 12:00 am is noon."  Or vice versa.  I can't really remember, but then that's the point.

I took a quick survey of the employees in our twenty-person company and determined that roughly 50% of us felt that 12:00 am was noon, and 50% felt it was midnight.  In the intervening years I've discussed this topic with plenty of other people, and I can confidently say that there exists no consensus on this matter in our culture.  That make sense, of course, because 12:00 am and 12:00 pm are nonsense words.  The true, meaningful words are noon and midnight.

But this product had an expectation for what these terms meant.  You could type in an alert for 12:00 am (or 12:00 pm) on a certain day, and the program would accept it.  And then somewhere along the line, the alert would go.  But according to my quick and dirty research, 50% of the time this alert would go off at midnight, therefore failing in its function to let you know that it was time for your noon appointment.

It was easy enough to determine which was which.  At 11:55 one day I set a pair of alerts called A and P, set for 12:00 am and 12:00 pm, respectively.  One or the other popped up five minutes later and I wrote that fact in big bold letters in the manual in the hopes that users would see it and notice.  But that seems a backward approach, now doesn't it?  A better approach would be for the machines to work the way the humans expect them to.

That's one pithy example, but this kind of thing was going on all the time.  I had become familiar in college with a concept in psychology they called human factors, and right around this time the software development community was rediscovering it under the name usability.  So I appointed myself usability manager and started proposing how to change the product to be more intuitive to human beings.  I did on-the-cheap testing by walking around with a yellow pad and watching people perform tasks in the product.  As the company grew I developed a department under me and eventually got to the point where I had a full time interface designer working for me.  He was much better at it than I had been.

During that time I postulated Tim Rules of Human-Machine Interactions, which state,
  1. Machines are here to serve humans and not the other way around.
  2. If humans expect a machine to behave in a certain way, and it behaves differently, then the machine is always wrong.
  3. Machines that defy the expectations of the target user are misdesigned, even if the actual creator understands how to use the machine.
  4. To the extent it is possible, it is more efficient to create a machine that works as target users expect it to than to educate target users to change their behavior.
In the intervening twenty years usability became user interface and then user experience.  The tools got more sophisticated, with multivariate testing and heat mapping and follow-home studies and discreet choice analysis and so many others.  But these four rules have always served me.  My conviction in them has not wavered, and I continue to use them to this day.

Friday, April 8, 2011

What not to say: Internal customer

Oftentimes business speak has the primary harm of being obfuscatory or simply silly - "I reached out across the cross functional matrix to create alignment of goals and synergies" rather than "I cooperated" - but sometimes there are bizspeak terms that are downright detrimental to organizational performance because they mislead or force the wrong behaviors.  One of those terms is internal customer.

In case you're lucky enough not to be familiar with this maddening phrase, internal customer means the employee or group of employees who will directly benefit from the work you do at the firm.  There is nothing inherently wrong with having a term to describe this relationship, but in practice it's a highly loaded term that I've only seen misused.  The problem is customer.  Customers are, of course, god to any company.  Sam Walton famously said,
There is only one boss.  The customer.  And he can fire everybody in the company from the chairman on down, simply by spending his money somewhere else.
I like that sentiment a lot, as do many people in successful companies.  And as a result this word customer becomes incredibly powerful.  The customer is always right.  It's what the customer wants.  Customers first.

But the phrase internal customer is a perversion of this powerful positive sentiment.  Customers by definition are not internal.  Internal people are part of our team and must all band together to delight the customers.  Customers are not part of our team and expect to be delighted, and if they're not, they'll leave us for someone else who does delight them.  The customer does not expect to sublimate her desires to anyone else's.  Internal people, on the other hand, don't have the privilege of expecting to be delighted and must constantly sublimate their desires to those of the customer.

At its best this phrase is misused by good-hearted people who accidentally credit too much importance to the wrong things.  At its worst, it is a deliberate manipulation pulled out by scheming people who seek to advance their own agendas at the expense of the company's greater good.  These people know how powerful the word customer is and know that they can create an environment where it is politically impossible to say no to them, and they do it by confusing themselves with actual customers.

In organizations I run, I ban the phrase internal customer.  There is no such thing.  If you say that phrase, I won't hear it.  My simple rule of thumb goes as follows.
If the party in question receives compensation from the company in return for goods or services rendered, it is not a customer.
Employees get paychecks and bonuses and stock options and health care and whatnot.  They're not customers.  Agencies and printers and vendors are not customers.  The people you rent your office space from are not customers.  Contractors are not customers.  Advertising sales reps are not customers.  That guy who comes and fills the vending machines in the break room is not a customer.

Resellers are not simple customers, since they make money selling our products, but they're not the same as pure insiders either, since they may be able to drop us and resell our competitors instead.  In other words, some days they're customers and some days they're not.

Now there are plenty of outsiders that are in no way customers or even prospective customers.  That's okay.  We don't tend to confuse them for customers, and they don't tend to pull power plays in meetings where multiple VPs are present.  So we can be less concerned about them.

Also be aware that the same company can be a customer and not a customer at the same time, depending on where in the organization you're interacting and what you're doing.  If the guy who fills the vending machines works for Frito-Lay, and if Frito-Lay is a major purchaser of your IT solutions, he still is not a customer, even though someone else at Frito-Lay is.  There are very occasional instances where the same individual sits in both roles, usually near the top of the organization, and we have to be cognizant of them, but that's pretty rare and usually obvious when it happens.

So does that mean this phrase is always unambiguously evil under all circumstances?  I suppose if you're in a purely internal function like IT or HR or facilities and you never, ever do anything that could directly affect an actual customer, then I suppose it's an okay phrase so long as it's confined to discussions that exist entirely inside that sandbox.  But if you're in sales, marketing, customer service, engineering, legal, finance, or senior management, forget about it.  You need to keep clarity on who is the customer and who is not.  For anyone in any of those roles, forget the term internal customer.  It'll only do you harm.

Monday, April 4, 2011

The Seven Habits of Highly Effective Marketing Departments

Marketing programs cover a lot of ground.  These programs are widely variable in such qualities as:
  • Expense range
  • Time to execution
  • Labor intensiveness
  • Predictability
  • Ability to be tied to ROI
  • Skill set required
  • And many more.
It can get pretty complex.  Therefore to facilitate planning and organization of our thinking about marketing programs, I use the framework I call the Seven Habits of Highly Effective Marketing Departments.  This framework captures and organizes the most basic marketing program types in use by high tech companies today.  It's meant to be an aid to thinking, not a proscriptive, dogmatic requirements document.  In other words, it may be that for the specifics of your marketing program you choose not to do one or more of these habits (or that your business has something special that doesn't sit on this list).  However, the bulk of high tech companies are covering this entire framework and doing little that falls outside it.  Therefore, it's a good tool for anyone who is creating a marketing plan or who is thinking critically about the program mix with an eye to improving it.

The Seven Habits of Highly Effective Marketing Departments are:
  1. Advertising
  2. Direct mail
  3. Events
  4. Public relations
  5. Web
  6. Collateral and sales tools
  7. Channel marketing
Not necessarily in that order, depending on the specifics of your business.   I'm expecting to get into a great deal more depth on all seven of these topics in the months and years to come, but for today I'll just define the categories.

Advertising.  Advertising includes all paid placements for your marketing messages in media or venues that someone else controls.  Advertising can be purely direct-driven, with a call to action that may even be asking for a sale, or purely brand-driven, or anything in between.  Advertising includes print, online, broadcast, outdoor, in-product, or others.  If you have a newsletter and I buy a blurb at the bottom of it, that's advertising.  If I sponsor Masterpiece Theater on the local PBS station and I get a five-second bumper before the show, that's advertising.  If I negotiate with a software vendor to build a link into its product that promotes my after-market add-on, that's advertising.

Direct mail.  Direct mail is any marketing program whereby I push my message out in mass to targeted lists that meet specific criteria I set out.  Direct mail can be physical "junk" mail, or direct e-mail, or communications using other mechanisms such as RSS subscriber lists.  Most direct mail is call-to-action oriented and intended to drive ROI, but that's an accident of what direct mail tends to be good at, not a requirement of the program type.  If you send renewal notices by e-mail to your service subscribers, that's direct mail.  If you send gift baskets to your large accounts every holiday season with no call to action at all, that's also direct mail.

Events.  Events are timely.  Events are run by the company or its partner during a specific time period for a specific target audience with a specific end in mind.  Events tend to be a little less direct-response oriented, but they don't need to be.  Events include trade shows, webinars, limited-time feel-good offers, and parties.  If you get a booth at CES, that's an event.  If you create and run your own trade show, that's an event.  If you take customers or prospects out to a hockey game, that's an event.

Public relations.  Most marketing programs are what we call controlled communications, meaning the company chooses exactly what will be said.  If you create an ad or make booth signage or write a sales script, that's a controlled communication.  Public relations encompasses all activities seeking to influence uncontrolled communications that are generally visible to the interested public.  Public relations includes press relations, investor relations, analyst relations, social marketing, and word-of-mouth marketing.  If you do a press tour and call on the New York Times, that's public relations.  If you run a Twitter feed, that's public relations.  If you comment on someone's blog, that's public relations, too.

Note that I have a strong personal habit of using the abbreviation PR.  In the high tech marketing world PR can sometimes mean public relations in the broad sense and sometimes mean press relations in the narrow sense.  I, too, use it contextually.  I'll be sensitive to being clear about which usage I mean at which time.

Web.  At this late date in history all technology sold to the public depends on the web, at least for distribution of marketing messages and typically for lead generation, support, and often actual order taking and fulfillment.  Web includes static site content, knowledge bases, SEO/SEM, and web analytics.  Companies can have multiple sites based on different geographies, brands, customer bases, and uses of the site.

Collateral and sales tools.  These are the marketing assets you create, either for direct customer consumption or to enable your sales team.  Data sheets, white papers, slide decks, and videos are all collateral or sales tools.  ROI calculators and product selection wizards are sales tools.  But so is a sales script or a few bullets or a key stats document.  Sales tools and collateral don't have to be fancy.  They just have to be effective.

Oftentimes I and others refer to the deliverables in this category as assets.

Channel marketingChannel marketing refers to a your reseller channel, if you have one.  Channel marketing can encompass all of the above categories.  In other words, you may need assets for your channel or a web site for them or you may direct market to your channel.  What's specific to this category is that it doesn't encompass programs aimed at the end customer.  Rather, channel marketing is aimed at your resellers and seeks to make them more effective.  Sales training modules for your own team are sales tools, but the same modules for your resellers are channel marketing.