themeat.org

Most PMs I interviewed at Google did not pass the test. You won’t either unless you show that you’re more than just a product manager. You have to prove that you’re a Google product manager.

—

I remember my own interview at Google pretty well. I knew that most people who interviewed there didn’t get hired, so I focused on my mental game. I went in with an attitude that I’ll just try to have fun with it — show that I’m excited to work there and that I care. Most of all, focus on enjoying the experience and trying to learn more about what it means to be a product manager at a company like Google.

Across a phone screen and two on-site interviews, I was subjected to hours of questions that I had never heard before with lots of hard, ambiguous problems to solve. It was difficult and stressful, and yet I managed to stick to my plan — stay happy, stay excited, show you care. I left thinking that I had a good time. It would be OK if I didn’t get hired. And if I got hired, it would be a pleasant surprise.

To my surprise, I got hired.

And within a few weeks, I was interviewing other product manager candidates.

It was daunting. I barely knew what I was doing at my own job, and they were already trusting me to make decisions about who to hire. I knew I’d have to level up my interviewer game to ensure that the next PM who gets hired was better than me.

By the time I left Google, I had interviewed dozens of product managers. Most of the time I was conducting phone screens. That meant I was the first product manager you’d talk to at Google. It was my sole decision that determined whether or not you moved on to the next round of interviews.

Out of the over 50 product interviews I conducted, I gave top scores to only 3 of them. And even if I signed off on you, you’d still have to get the approval of all your other interviewers, the hiring committee, and company executives. It’s a huge feat to get hired by Google.

Why is it so hard? Mostly because we’re told very clearly that we should only hire people who make Google better. Most PMs I interviewed did not. It’s not that they’re bad PMs; it’s just that they’re not Google material.

Here are some specific reasons why you didn’t come off as Google PM material in your interview.

You’re not user focused

I always enjoy asking open-ended questions like, “if I delegate this problem to you, what would you do?” Many people answer they’d go build some solution or dig into data. Those are the wrong answers.

The best answers are always about understanding people and the underlying problems they have. Tell me you’d go do some interviews, conduct surveys, talk to experts, or somehow get to the heart of the problem. Then you can get to the data analysis and other bits. I was always stunned about how frequently I’d get an answer like, “I’d go build a new app…” without any attempt to dig into a problem first.

Show that you really care for and empathize with people who use your product. Prove you’re willing to go the extra mile to understand their pains and problems. This is the heart of product management at Google and anywhere.

You weren’t excited and confident

Most people who interview at Google are nervous — and that’s totally understandable. You’ve been given this one shot to get a job at one of the best companies in the world. You don’t want to fuck up, and that makes you jittery and anxious.

When nearly everyone is like that, I’m always pleasantly surprised when I get that candidate who’s happy, engaged, and enthusiastic to be there. They usually have a little smile on their face. The tone in their voice shows their happiness. They’re clearly excited to be here, and that makes me excited to be there with them. It shows they’re confident and ready to handle any problems might come their way, even though they might be stressed right under the surface.

So channel that anxiety and nervousness into confidence and joy that shows you came here to get this job. It helps if you go into the interview as if you have nothing to lose — you don’t because you probably won’t get the job anyway. If you embrace this feeling of excitement and happiness, you’ll amaze yourself how different your answers and behavior will be during the interview.

You didn’t communicate clearly

“Product manager” is a fancy term for “communicator” because most of what we do is listen, explain, and negotiate. Because of your nervousness and stress, sometimes it’s hard for you to think clearly. Your brain is firing off all kinds of thoughts, you’re not sure which way to go.

Then you try to speak. What comes out is a lot of words — a stream of consciousness, not an answer. I’m sitting across the table from you struggling to reassemble your words in a way that I can understand. You have a picture in your head that’s not developing in mine because your brain is getting in the way.

The best way to handle this is to take a few seconds before you answer to plan what you want to say. Then focus all your effort on clearly communicating those concepts. Make sure you answer the question I asked, and make sure to explain why you chose that answer. Don’t think of new things to say while you’re talking because that will get in the way of your clear communication.

You weren’t Googley enough

I always asked the same questions in my interviews. It made me more consistent as an interviewer, and it gave me a baseline that I could use to measure you against. Because of this, I could tell in the first few minutes of the interview whether or not you’d pass the test.

Most people’s answer’s were middle of the road. They lacked that creativity that shows they were masters of tech, design, and strategy. The candidates didn’t show their enthusiasm about applications of new technology or the brainpower that’s available to them at Google. Their tone of voice was flat or nervous, and they looked like a deer in headlights. They gave answers I had heard before — nothing novel or interesting. It was boring to be with them. I wanted to get out of the interview as soon as possible and even less to be their coworker.

“Googley” is the term that’s used to describe the intangible factors that separate out the people who are really enthusiastic about their work and working at Google. It’s a combination of being knowledgeable, passionate, humble, and friendly that embodies what an ideal Google employee should be. In short, I should be excited that you’re going to join the company — that you’ll fit in with everyone, that you’ll be fun to get lunch with, and that you’ll do it with zest.

This is really the hardest of all to advice about because it’s so subjective. First, you need to do the things above — prove that you care about people and their problems, show your excitement and engagement being here, and clearly communicate what you’re thinking. Then you have to go the extra step to prove that you really know and love product –that’s the best you can do to show that you’ll fit right in as a Google PM.

Coda

There’s a part of my Google interview history that I neglected above… That was my second time interviewing at Google.

My first interview with Google was three years earlier. I managed to pass the phone screen and get an on-site interview. And then I fucked up. I was nervous. I couldn’t think straight. I gave terrible answers. I walked out knowing that I didn’t make the cut, and I was proven right.

A few years later, a Google recruiter got in touch with me asking to come back and try again. I said yes, and I went in with that attitude I mentioned before. I probably won’t get hired, so I should embrace the experience so I can stick to my core values instead of fighting against my nervousness and anxiety.

To my surprise, I got hired.

I got lunch with my new manager a few days before my start date. He said, “do you remember me?”

I said, “no.”

“I interviewed you when you came on site a few years ago…”

“Oh god.”

“…and when I saw your resume come up as a new hire, I went to the HR team and asked them to put you on my team right away. You’re exactly the kind of PM that I was looking for.”

It was that moment that I realized I had really passed the test.

You can do it too.

Great product teams know why people love or hate their products, not if people would recommend them to friends.

NPS – net promoter score – is a metric that many companies use to assess product satisfaction. I’ll assume you know how it works, but if not, see this footnote for a quick primer.

Give me a few minutes of your time to prove to you why NPS sucks and how you can measure love instead.

Why NPS sucks

It’s trademarked
NPS isn’t some commonly accepted measure like temperature or length. It’s a number invented by an agency and it’s been trademarked. If I was an agency and I was looking to drum up work, I’d invent a loaded metric that always shows your product needs improvement, lock it up with intellectual property laws to prevent others from using it, then I’d sell you a solution that optimizes just that metric. Let’s dig into how loaded it is.

Ignoring 28% of the results
I swiped a ton of NPS data from indexnps.com – a site that has lots of NPS results for a wide swath of brands. Recall that responses of 7 and 8 are ignored in NPS results, yet 28% of NPS responses fall in that range. If I was an agency trying to get a client, throwing out a quarter of positive results without any justification is a great way to stack the deck in my favor that you need my help.

Biased results
People often answer survey questions based on what they think you want to hear rather than give an honest response (“experimenter effect“). In the case of NPS, they’ll answer that your product is great even if it isn’t. A full 67% of the indexnps scores were 9-10. Were those people really excited to recommend those products or were they trying to satisfy the pollster?

What are they recommending?
The people answering your NPS question could be recommending any part of your product. For example, if your website has an engaging user experience and separately a monetary incentive for participation, people could be recommending your monetary incentives (“Hey, get free money here”) over your user experience (“Hey, this site is really engaging”). This is an essential detail that gets paved over with the overly simple NPS question.

Hypothetical vs. concrete
Since NPS is framed hypothetically (“Would you recommend…?”), you’re not learning if people actually recommended it (“Did you recommend…?”), to whom they recommended it, how they recommended it, and whether or not that recommendation actually had an effect. NPS leads to more questions than answers.

It’s not an outcome
Companies treat NPS as a key outcome — ex. “Our goal this quarter is to increase NPS 20%” — without understanding how or if that translates to other business results. NPS isn’t an outcome, high scores doesn’t ensure more business, and it doesn’t tell you if customers will stay loyal to your product.

Measuring Love

My hypothesis is that love and hate are the best indicators of your products’ future success, so you need to measure them.

Humans make decisions based on emotions (then justify their emotional decisions with reason). Your goal is to make a product that triggers positive emotional outcomes so that people become loyal to your product without realizing it. Negative emotional responses and decreasing positive emotional outcomes are signs that you’re going to lose customers.

Ask these questions instead to learn whether or not people love your product.

“How do you feel about the product?”
Have the person choose from a series of smileys with a range of emotions – elated, happy, neutral, angry, crying, sick, love, etc. Keep track of the positive versus negative emotions and how they change over time.

“What do you love most about the product?”
Sure, it’s a naive question, but it bluntly gets at the issue. These are the items you should feature in your marketing materials and sales pitches to get future customers’ attention. Ask it open-ended (vs. multiple choice) for best results.

“What do you hate the most?”
These are the problems that you need to fix or you’ll lose customers. Note that different types of users will have different problems, so you need to ask a wide swath of users. For example, if you’re making a business app, the everyday user will hate some parts (ex. “this page sucks”) versus the ones that the buyer hates (ex. “this costs too much”).

“Have you recommended…”
Ask this yes/no question – “Have you recommended [my product] to a [friend/colleague/etc] in [the last month/year/ever]?” Try to learn about the context of the recommendation and why that person recommended it. Concrete questions are way better than hypothetical ones (“Would you recommend…”).

“How did you recommend…”
Ask how someone explained your product to another potential user of your product. Your everyday users can explain your product to other potential users better than you can. You need to understand the language of your users, then use those words when talking to other prospective customers.

Measuring love is hard and necessary

The promise of NPS is that it’s a quick way to assess how well your product is doing — that recommendations are a proxy for product success. Taking shortcuts with something as complex as social interactions or emotional reactions can lead to worse results than doing nothing at all.

This gets to the core of the matter — a great product team should always seek to understand why people love and hate their products, and they should be be able to tell you why without hesitation. “Hey PM, what do people love the most about the product?” If your product team can’t answer that question immediately with concrete examples, they’re not doing their job.

So do the right thing with your NPS scores — get rid of them — and instead learn how people feel about your product. I’d take one person who loves my product over a hundred who’d recommend it to a friend.

—

tl;dr on NPS: NPS is usually asked as a question in a survey like: “From 0 (not at all) to 10 (definitely), how likely are you to recommend [my product] to a [friend/colleague/etc]?” The NPS score itself is (% of people who answered 9 and 10 – % of people who answered 0 to 6).

So if your nps is 70% (that is, there’s a 70% difference between people who answered 9-10 versus 0-6), the general interpretation is that people like your product. A score of -20% (that is, the percent of people who answered 0-6 was 20% more than people who answered 9-10) probably means that people don’t like your product.

“Is it just me or does every product manager feel like this?”

My former, product manager coworker asked me this while walking the streets of Manhattan. He had recently gone through some work shit and was struggling with it — experiencing the doubt that strikes at the heart of anyone in a tough work situation. He asked that question rhetorically, as if he expected the universe to give him an answer so he could resolve his doubts and move on.

I stepped in for the universe. “It’s not just you. We all feel that way.”

Product management is a difficult, burnout-inducing profession. We do it because we love the idea of it more than actually doing it. I’ve hit the point in every product job I’ve ever had where I tell myself, “this is the last product job I’ll ever have.” Still, I keep doing product work, hoping that the next job I have will be the one that fulfills my expectations.

What will it take to close the gap between our expectations and reality in the product management profession?

—

Let’s unpack the job of a PM. We take an ill defined idea and turn it into something that people will love.

That means we need to understand what people need, lead a team to create an amazing solution, ship it into the world, see what happens, and then start the whole process over again. If all goes well, the people who use the product pay us (ideally with money) in return for how much they love the product. The job sounds pretty great.

It sounds pretty great, in theory.

In practice, a product manager struggles to balance the needs of:

• the company who depends on the product’s success
• the customers who pay the bills
• the users (who may or may not be the people who pay)
• executives and others who impose their demands
• new problems which have to be dealt with ASAP
• themselves and their happiness

That last one is the hardest by far. We product people are unhappy. That’s more than just my opinion.

“Product manager” was named one of the unhappiest jobs for two years straight, 2011-2012. Don’t believe it? Ask a PM about his/her job and you’ll see the sadness in their eyes. I can’t remember the last time I met a PM who was truly happy with his or her job.

Why are we product people so unhappy? The explanation is deceptively simple. There’s an enormous gap between the expectations of what we want in a product job and the reality of doing the job.

Let me describe the gap in more detail.

You expect:	ownership of the decisions and direction for a chunk of the product
In reality:	you’re expected to adopt the same decisions and opinions as your boss & managers

You expect:	authority to make decisions based on hard data, user needs, and other evidence
In reality:	HiPPOs (Highest Paid Person’s Opinion) make decisions despite or withholding evidence

You expect:	to work with a close knit team to focus on delivering that product
In reality:	you can’t deliver because the team and priorities change too often

You expect:	enough support to make the product as successful as it can be
In reality:	you capitulate because you don’t have the resources to make it successful

You expect:	responsibility to make the decisions that make the product successful
In reality:	you have to get layers of approval, and your decision can be overturned by anyone above you

You expect:	recognition if things go well, or blame if they don’t
In reality:	no recognition when things go well, and all the blame if they don’t

You expect:	bottom-up management — you tell the execs what they should be paying attention to
In reality:	top-down management — someone else tells you what you’re going to do next

You expect:	a job where you’re the fearless leader of a team who fights to achieve awesomeness
In reality:	you’re a middleman who takes the specs from the customers or managers to the engineers

That gap between reality and expectations is called burnout. The larger the gap, the faster you and your team will descend into underperformance and depression. And that, in a nutshell, why PMs are so unhappy.

Despite all that, we still do the work — hoping that the next job will be the right one where we can do product management “the right way,” where our reality will match our expectations. And if there is a gap, hopefully we’re paid well enough to ignore that gap.

But that’s a horrible capitulation. There’s got to be a better way to do this.

—

What does product want?

We want to…

• make something that people love
• own our customers’ and users’ goals and problems
• make decisions based on data and evidence, not opinion
• have the authority to define our own success
• have the responsibility to achieve our success
• get the resources we need to reach that goal
• have the trust of our stakeholders and managers

That last one — trust — is the hardest one. “Why should anyone trust the product team?” I’ve heard that said too many times and too many ways in my product management career. “First, product has to earn the trust of the organization. Then they can take ownership.”

Therein lies the problem. A healthy team is built on trust — trust that the product team knows how to do their job, how to understand users, how to work with stakeholders, and how to make the right decisions that will make the company and product and users successful.

A healthy team trusts by default.

When trust is lacking, you’ll get the “in reality” situations I described. HiPPOs tell the product team what to do because they don’t trust the product team to make decisions themselves. The company’s course changes constantly because nobody trusts in the product team’s direction. The product manager is blamed for failure even though the product manager never even made a decision by his/herself.

Fixing the problem begins with trust.

To all you executives, product management leaders, or anyone who interacts with your product teams — if you want a healthy product team, you have to trust them with the authority and responsibility to do their jobs. And you should be uncomfortable to cede that much power to them.

It’s a difficult request. Product management is a critical role in a company, and everybody has an opinion on what’s the next most important thing to produce. Giving that power away is a difficult decision, but it’s one that’s essential if you want to see a product team thrive.

The best product team I ever worked on was built on trust. Each PM had a well defined role and knew how their product area laddered up to the company objectives. We were held to goals that we defined for ourselves, and we were given the authority to achieve those goals as we saw fit.

The worst teams I’ve been on, on the other hand, were built on all the opposite. The management questioned and trumped the teams’ decisions regularly. There was no clear ownership and no authority given to the product teams. The team was told what their goals were. There was no trust, and no path to achieve that trust.

If you don’t trust a team or are so uncomfortable that you won’t cede that power to them, fire those people and replace them with people who you do trust, with people who you would empower with real ownership.

Because if you’re not willing to trust them, you don’t have a product team. You have a bunch of overly-paid people who pass specs from the managers to the engineers.

So make the decision to trust your product team. Work with them to understand the problems facing your organization. Help them come up with amazing solutions to those problems. Believe in them when they ship their solutions. And celebrate their wins to show your thanks.

That’s what product really wants — trust. And that will make your product managers happy.

If you’re selling experiences, copies and forgeries will not devalue that experience if your experience is truly worth experiencing.

—

Believe it or not, the following pictures are illegal, and you definitely shouldn’t see them (and I definitely shouldn’t have them).

Many years ago, I saw Spamalot, the Monty Python musical. Immediately after taking the photo of the intermission curtain, an usher rushed over to tell me what I did was against the law and insisted that I delete it right there in front of him. (Obviously I faked deleting them.)

Yes, those crappy, crappy photos were taken illegally. Seriously, cell phones are banned in NYC theaters. But what’s the point of prohibiting photos when it’s the experience of being there that’s valuable?

Here’s another photo of questionable legality, taken at the Museo de Arte de Puerto Rico.

Before I took this photo, the docent caught me with my camera out. He said, very politely, “please no photos of the art here,” and disappeared. A moment later, he peeked his head around the corner. With a smile, he said, “well, you can take that one,” then disappeared again.

It’s as if he secretly wanted me to share the art but had his hands tied, so he let me off the hook just this once. I only wish the photo had come out better to do him proud.

Finally, here’s one of several I took at the David Byrne/St. Vincent tour a while back.

Before the show began, David Byrne came over the PA and said, “we encourage you to share photos and videos of the show because we’re proud of we’ve done.” You can find thousands of videos uploaded to YouTube of their wonderful concerts. It was a fantastic show. Do yourself a favor and check out any of their renditions of Road To Nowhere.

—

It all comes down to ownership and fear.

For the person who “owns” what I’m taking a photo or video of, they’re afraid that my photo or video will devalue the real thing. Spamalot will lose a patron because he saw a video of it on YouTube. The San Juan art museum will get one less visitor because she saw that print in my photo. David Byrne will sell one less ticket because the whole tour is on YouTube.

They’re all unfounded fears.

I record my experiences for a few reasons — to remember what I’ve done and where I’ve been, to brag about what I’ve done, and to share a piece of what I experienced. Nothing can (yet) reproduce the amazing experience of “being there,” and so photos and videos are the next best thing.

It’s not the “real thing,” so why put handcuffs on what gets shared? For example, after the very first performance of Spamalot, there are no more secrets. Word will get out about how good or bad the show is. Video and audio will leak. The public will determine its success by buying tickets. I saw it twice because I loved it the first time and wanted to share the experience with a friend. If my second viewing was via cellphone video instead, it wouldn’t be the same.

So ask yourself what’s really precious. If the experience of being there in person is the valuable part, then it doesn’t matter what people share about it. In fact, sharing can encourage more people to experience it in person.

Coda

A friend asked me for advice. He was wondering who should be allowed to attend the launch of his Awesome New Thing. Specifically, he was concerned that some of the attendees might try to steal his ideas for their own.

I said, “If your ideas are good, they’ll absolutely steal them. The question is — what is it that they can’t reproduce? What makes your experience unique?”

Because if you’re really onto something, the best validation you can receive is when someone shares the experience with someone else or when a competitor imitates you. So why delay the inevitable? Encourage it instead.

*Yes, I’m aware that photo taking can impede memory. Let me enjoy my ignorance.

I prefer to do my research before leaping to outrage.

Of course, I’m referring to the recent revelation that Facebook manipulated users’ news feeds to see if it had any emotional impact on them.

It’s worth asking the question… just what the hell did Facebook experiment with? And why did they do it? And should I really be mad because of it?

If you’re wondering what my credentials are to offer this analysis, there’s not much. I’m an armchair cognitive and social psychologist, so studies like this always pique my interest. So I’ll try to be a bit silly (though not as silly as usual), and I’m certain to skip over lots of relevant details, and hopefully you’ll learn a thing or twenty along the way.

Background

Recently, social sciences are in the midst of a fad studying how emotions and other things spread in social groups. The fancy words for this are “emotional contagion” which reflects how emotions can be shared among people verbally and non-verbally.

I first heard about this from the Framingham Heart Study which showed how obesity and happiness spread through family and social connections. If you read the details, you’ll learn that you’re at an an increased likelihood of gaining weight if your friend, spouse, or family member gains weight. The same study shows happiness spreads through those ties in a similar, “more likely” fashion.

Further studies have shown that yes, emotions are contagious. For example, this effect has been seen in lab settings. Researchers in China studied Weibo posts to show how anger spreads faster and better than happiness or sadness in social networks. If you’re a believer in evolutionary psychology like me, then you’d say that empathy evolved to maintain healthy social groups (read as: mates).

By contrast, some people have argued that simply using Facebook makes us less happy, regardless of what content we’re exposed to. In her book Alone Together, Sherry Turkle argues that replacing rich, face-to-face interactions with computer-mediated communication leads to an illusion of intimacy; the convenience of communication technology actually causes us to be more isolated. (Yeah, this is a slapdash summary. Just read the book.)

There’s some scientific research to support her perspective. A University of Michigan study found that Facebook use predicted a decline in a person’s well being. They polled students throughout the day, asking for their mood and how much they used Facebook since the previous polling. They found that the more someone had used Facebook, the worse their mood would be. Sadly, the press misinterpreted this as “using Facebook makes you unhappy” because they don’t understand the difference between correlation and causation.

We get it. What did Facebook do?

Most experiments to date have looked for correlation between an emotion in a social network and that same emotion in the experimented individual. Facebook sought to answer a variant of the question: if you decrease seeing a certain emotion in a social network, does it result in an increase of the opposite emotion in the individual?

I’ll spare you the brain-numbing experience of reading the paper with this hasty summary. Facebook with help from some folks from Cornell tested two conditions — whether seeing fewer positive posts would lead to increased negative postings, and whether seeing fewer negative posts would lead to increased positive postings. They randomly picked 600k Facebook users from the FB database for the study.

How did they test this? Facebook manipulated the news feeds those users, showing them fewer happier or fewer sadder posts based on words which matched those emotional outcomes. Then they checked those users’ subsequent posts to see how the emotional content changed. The experiment lasted a week.

It worked. People exposed to fewer positive words expressed more negative words in their posts and vice versa as compared to control groups. The researchers noted that this effect was greater when positive posts were hidden than for the group who had negative posts hidden.

Another result is that this showed how emotions can be affected non-verbally, independent of face-to-face contact. Are the emotional outcomes in our own Facebook posts because of content we see in Facebook or do they reflect, for example, emotional outcomes in our face-to-face relationships as filtered through our posts? This seems to demonstrate the former, that Facebook posts alone can affect our emotional state (in our posts).

Also, people in the experiment group were less expressive overall, while people in the control group (who saw greater emotional content in their posts) used more emotional terms in their own posts. The researchers posit that this rebuts the arguments by Turkle and others that using Facebook makes us less intimate with others.

And here’s the part where you get mad

Reason number one to be mad — there was no need to manipulate users’ feeds to prove the point. Instead, they could have run a lab study to see how the decreasing emotional input affects your emotional state. Such a large scale study with all the data manipulation was unnecessary. I wonder if that occurred to them before they started this study.

Number two — the experiment worked. Kinda. In the various experiment conditions, the percent of positive or negative words budged 0.1% at most. That means this change affected the emotion of 1 out of 1000+ words posted on Facebook. That’s not much of a change. Even if it’s a “statistically valid” result, I wouldn’t brag about it. Also, that change doesn’t mean users were happier or sadder. It only proves that they changed their emotional word choice.

Getting angrier, the researchers stretch to shoot down arguments that using Facebook makes you less happy, like those in Turkle’s book or the uMich study I cited earlier. But that’s comparing apples to oranges. Facebook was checking post content days after the experiment went into effect. The Michigan study was checking on the emotional states of people a short time after checking Facebook. Turkle’s book was ethnographic (interviews). You can’t refute one with the other. This only leads to more questions about why these research results are so different, or why they needed to conduct an experiment of this scale to prove the point.

There’s also the question of researcher ethics — specifically that they went forward with a research plan that you intuitively know would make a group less happy as a result. Honestly, I’m not that bothered by an experiment that would make people happier as a result; we could use more of that in the world. Cornell held the experiment at arm’s length, saying that their researchers “did not participate in data collection and did not have access to user data”. Seems rather disingenuous to wash your hands like that when you could be all but certain of the negative results that some in the experiment would face.

So who in Facebook should have stopped this? Nobody. As XKCD perfectly noted, Facebook would have done this research anyway. And they probably have done this research before but never made the results public. If the results of this experiment lead to people staying on the site longer or clicking more ads, then it was worth it to Facebook.

This leads to another point. If you’ve read the paper, you might have noticed that the most interesting results are missing. Did users in the experiment group come back to Facebook more or less often? Did the experiment had a positive or negative effect on Facebook’s bottom line? Were there any long term effects? In other words, was it worth ($$$) it for Facebook? The paper is painfully silent.

Epilogue

This experiment is just a drop in the bucket. We’re the subjects of hundreds of online experiments on a daily basis. Google might be the most notable of the experimenting bunch (for example, testing 40 shades of blue to see which performs best), and other companies have followed Google’s lead hoping this ruthless pursuit of optimization will lead them to similar success. However, few if any of those studies are made public, and even fewer get the attention that Facebook’s study did.

So why did this experiment in particular trigger so much anger? My hunch is that the outrage lies somewhere between “can we really trust Facebook?” and “fear of missing out on what my friends wrote.” It’s the same outrage that underlies big data’s ability to manipulate our lives which danah is all over. We’re quite helpless against the machinations of their algorithms. And thanks to that research I cited, we know that the anger about this issue can travel very quickly via social networks.

Even if Facebook is manipulating your emotions, then so is Google’s search algorithm, that game you just played on your iPhone, and your circle of friends. To have long term effects, you’d need consistent exposure to the same emotional outcome over long periods of time, in the same way you get lung cancer not from one cigarette but from years of smoking. Facebook is just a small part of your total emotional well-being. Emotional reciprocity over long periods of time is a much more important part of that well-being.

That’s why I see no harm — and lots of potential good — by letting people choose to be part tests like this. Why not let people opt-in to a Facebook experiment to prove that you can be demonstrably, permanently happier by manipulating Facebook posts for years? Add a “Make Your Facebook Happier” button. I could do with fewer curmudgeons and negative posts in my news feed. I’m sure lots of others would agree.

Look, I’m sympathetic about what happened here. There’s never a straightforward answer when trying to balance the needs of your audience with the needs of your employer. I’ve had to make many ethical decisions about the products I’ve managed in my time. You make the best choice you can, apologize if you make a mistake, and keep moving. Even if the decision seems significant, it’s usually small potatoes in the end. Time will pass, people will forget.

So what’s really important? The people around you. If you remember only one thing from this rant, make it this. If you want to change yourself — emotionally, physically, anything-ly — the best thing you can do is to change the people around you. Facebook posts, violent video games, and how much porn you watch are small potatoes compared to the company you keep.

As Seth Godin said:

Who you hang out with determines what you dream about and what you collide with.
And the collisions and the dreams lead to your changes.
And the changes are what you become.
Change the outcome by changing your circle.

A coworker came up to me. “I’m sorry for snapping at you yesterday after that meeting.”

“Huh?” I stalled. At that time of the morning, I was not fully conscious.

“I hope you weren’t upset because of it.”

Ah, that. My brain shook off its slumber to dredge up the memory. We were discussing a problem. My answers weren’t good enough, or the questions weren’t clear. It doesn’t matter. The result was — my coworker gave up in a fit of frustration.

“It’s ok. We’re all under a lot of stress right now,” I replied. I was being honest. The incident had barely registered on my radar. Like I said in my response, I’m sure it was due to circumstances and not because my coworker is a short-tempered asshole. (For the record, my coworker is not a short-tempered asshole.)

“I –”

“Hey, got a second?” Another person interrupted us. My apologetic coworker shuffled off.

About a minute later, I realized I had fucked up accepting the apology. And that got me thinking about the nature of apologies and taking responsibility for problems.

—

The subject line of the email read, “Important Information – Unauthorized Intrusion.” It piqued my interest. Either it was elaborate spam or a poorly written subject line that was actually serious.

I opened the email.

IMPORTANT INFORMATION. PLEASE READ IN ITS ENTIRETY.

Dear Patron:

We regret to inform you that on XX XX, 20XX, Company, Co. detected an unauthorized intrusion into its systems.

“Unauthorized intrusion…” You got hacked? Wow. I am soooooo regretful that you got hacked.

Why the hell should I care?

The answer to that question was at the end of paragraph one, in bold, because really important information should always appear at the end of a paragraph in bold. “Your information may have been involved in this incident.”

You can almost see the CEO holding this smelly apology at arm’s length, pinching his nose as he reads it aloud. He buried the lede in an attempt to soften the impact. My personal information and credit card number may have been stolen, and it’s his company’s fault.

That last sentence should have been the email’s entire first paragraph. “Your personal information and credit card number may have been stolen, and it’s our fault.” And how should he have written the subject line?

“We fucked up.”

—

I cringe every time I read something like “I regret to inform you.” Trite apologies are so… trite. “I regret to inform you that your dog is dead.” That’s believable when said by the veterinarian who tried to save my dog’s life, but it doesn’t work if you’re the person who killed my dog. (“I regret to inform you that I ran over your dog.” — see?)

Same thing applies when your website gets hacked. If my info is stolen, then it’s your fault, not the hackers. “Regret” is not an appropriate response. Instead, take the blame — as in, “this was my fault” or “we messed up” or “I broke your guitar when I threw it onto the plane.”

Fall on that sword.

And then apologize. And apologize with a sincere, personal apology. “I’m really sorry.” Not one of those corporate, dismissive apologies: “we apologize for the inconvenience.” “The” inconvenience — not even “your” inconvenience. How can I believe your company’s apology when you can’t admit that “the” inconvenience happened to a person?

When appropriate, offer a remedy. “Here’s a gift basket of smoked meat products and local hard cheeses because we really love you.” Local hard cheeses wouldn’t make me feel better about my wrecked guitar, but might make me feel better if it appeared in my hotel room after you’ve fixed my botched reservation (and after giving me a free upgrade).

What’s the worst thing you can do? Blame others. For example (actual quote here) — “a third-party criminal actor used hacking technologies to access our databases and may have accessed your personal information.” Fail. How about… “Our inadequate security allowed hackers to access the private information which you trusted to us.”

Don’t blame others, and don’t make me feel like I’m the one who’s wrong. “I’m sorry, but there’s nothing I can do for you.” That little act of disowning the blame puts it squarely on my shoulders. There’s plenty that you can do for me, but you choose not to do it. Liars 😛

Get the picture? There’s truth, and there’s truth. One is sincere, and one is not. One is “I did not have sexual relations with that woman,” and one is “I have had sex with women who work for me on this show.”

If you can’t be honest in your apology, then do us all a favor and don’t apologize.*

—

Finally, when someone genuinely apologizes to you, accept the apology with grace.

This person feels like shit about what happened, has no idea how you feel about it, and wants to apologize to make both of you feel better. The worst thing you can do is dismiss this act of humility. That’s where I fucked up with my coworker.

“I’m sorry for snapping at you yesterday after that meeting.”

“Thank you.”

—

* Don’t even get me started on, “we take your security seriously.”

The death of Steve Jobs is a mirror into our own vanity. Designers — take heed.

Steve Jobs blah blah visionary blah blah design blah blah changed computing. It’s true, and I’m not going to rehash the accolades here.

But there’s a flip side to the innovation. I wrote this as a Facebook status following his death:

Steve Jobs in hell is working 16 hour days 7 days a week at a Chinese factory building the new iPhone 666.

It’s funny only because there’s a dose of truth to it. In all the technology we acquire, not just from Apple, much of the cost that went into creating those artifacts is hidden from our eyes unless something brings it to our attention.

Besides dead factory workers, we pay the cost of Apple products everyday. Have you ever gone to the Genius Bar without an appointment? Broke your iPhone and had to buy a new one? Or had your macbook stolen only to have the police shrug?

Some more jokes at dead Steve Jobs’ expense to drive the point home:

• Steve Jobs in hell just had his iPhone stolen, and the police won’t do shit about it.
• Steve Jobs in hell broke his macbook, so he went to the genius bar without an appointment where he’ll wait for his name to be called… forever.
• Steve Jobs in hell broke his phone but can’t afford a new iPhone, so he had to settle for a free Nokia.
• Steve Jobs in hell got a $5,000 cell phone bill this month because he’s roaming on his data plan.

The jokes practically write themselves. Let’s face it — Apple products are expensive and fragile. They are time sinks and pickpocket bait. And we suck it up because the products are so tantalizing — because owning an Apple product is a statement about you.

If you think I’m denigrating the name of Steve Jobs or Apple, I am, but only as much as it takes to make my point. If it makes you feel better, you could play Mad Libs with the text above and swap out the names of the companies, products, and executives; it would be just as true but the products wouldn’t look as good. Simply put, we pay more than money for the privilege of owning these magical amenities.

Steve Jobs was responsible for Apple’s meteoric rise these last decades because there was an audience desperate for Apple’s pretty products — me included. If there’s any lesson to take from Steve’s passing other than a life cut off in the midst of greatness, it’s that we greatly value the veneer of our possessions despite the costs.

You might think your team does “agile development.”  You’re not agile.

At Infocamp Berkeley this weekend, I ran a session called “You’re not agile.” I fully admit it was inspired by my anger from organizations who claim they’re “agile” when they’re really not.  Here’s an alternate take on what I presented…

A long time ago, waterfall development was the process used to create software.  Executives would make requirements, designers would make designs with the requirements, developers would build software from the designs, QA would test the software, the software would ship, and the cycle repeat.  Software development was treated like factory work because that was the only model of production they had.  Replace “software” with “bicycle” in that explanation and you’ll see what I mean.

However, waterfall isn’t ideal for software.  Requirements change, customers have limited input, and software simply isn’t hardware.  Using waterfall for software is like cooking a souffle using directions for grilling a steak — worst souffle ever.  Similarly, using a process that works great for manufacturing physical goods is a bad idea for producing software goods.

What is agile development?

As a backlash to waterfall, teams began experimenting with new methods.  These groups started with the premise that there must be a better way to produce software in a corporate environment.

For example, waterfall process is obsessed with documentation as the interface between steps of the process.  However, producing functional software is a far more important business goal, even if you skip writing everything down.  Similarly processes and tools are useful, but you can solve problems with communication way faster than following a checklist.

These values and principles are codified in the Agile Manifesto — a document well worth your time to read and understand.  It was written by a group of agile practitioners looking to forge a group identity for their new vision of software development.

Sadly, it lacks a concise definition of what “agile development” means, so here’s my take:

Agile development is a set of values and principles for improving your development teams’ alignment with your organization’s needs.

That’s a dense sentence.  Read it a few more times to make sure you get it.

Take notice that my definition and the Agile Manifesto do not say anything about specific practices that will make you agile.  Rather, agility is a shared mindset that you and your team use to solve development obstacles.  A team with this as a motivating goal is a powerful force for creating kick-ass-ness in the world.

The cargo cult of agile

News of successful organizational transformation with agile development methods started spreading.  Success breeds imitators, and other teams tried agile for themselves.  They started doing agile practices — like daily standups, iterations, and product backlogs — but never gained the benefits.

They became the cargo cults of agile.

The term “cargo cult” refers to a group who imitates another group’s processes but lacks the substance which makes those processes work.  The term was inspired by the actions of Pacific island inhabitants in World War II.  U.S. and Japanese troops would set up bases on these islands.  With these bases came regular shipments of goods, visits from planes, and formation marches by troops.

After World War II, the troops, planes, and bases left too, leaving the native inhabitants without their regular deliveries of goods.  To bring back those deliveries, some of them recreated the artifacts and actions of the visitors — building airports with dirt roads, airplanes and radios from bamboo and coconuts, and marching in formation as best as they could.

Sadly, the natives were unsuccessful in making additonal cargo drops reappear because they didn’t understand the visitors’ airplanes, airports, cargo, and radios.

Similarly, many organizations put agile methods into practice without adopting agile values.  For example, a manager orders the engineering team to start doing daily standups. The engineers perceive this as “managers are spying on what we’re doing” and treat it accordingly.  By contrast, an agile team knows that standups are for the team itself — and not for the managers — because the team values communication, helping each other, and self-organizing.

Because the cargo cult agile teams don’t internalize the changes their making, they don’t get better.  As a result, there’s been a backlash against agile development from the people who’ve had bad experiences where “we tried to do agile, but it just didn’t work.”  They give agile development a bad name.

Lessons learned

In my session, I went over a few agile practices and how they represent the values and principles of agile development, but I’ll leave it up to you to find additional resources on how to be agile.  Instead, I want to touch on a few motifs that ran through the conversations and questions I heard at Infocamp:

• Agile UX — How does design fit into agile?  In short, agile UX means “stop drawing pictures and just start building.”  Organizations are very uncomfortable with this; many UX designers don’t have coding skills to build prototypes, and managers and executives are wary about committing development resources when designs are incomplete.  It will take a real sea change to make this catch on.
• Becoming agile — How do you begin agile development in your organization?  It takes lots of things — a willing development team, executive buy-in, strong values, reasonable tools, and a great agile leader to guide the team.  But what’s even more important is this — start small.  Do one project “agile.”  Remove organizational impediments one by one.  Bring in new practices little by little.  And one more thing — don’t be afraid of failure; just keep trying.
• Management skills — Most schools don’t teach agile.  Hell, most schools don’t teach management skills at all.  The people I know who do agile well learned it themselves or from someone who kicks ass at agile.  In other words, agile development needs better marketing and education to spread the word and knowledge.

Another way to paraphrase the questions is this — “How can I learn more about agile and apply it to my organization?”  I’ll give you three next steps:

1. Learn about agile development.  Read blogs and books about it.  Find people who know about it and pick their brains.  Go see a kick ass agile team and how they work.
2. Decide what you value.  Then start looking at how your team runs. What are you good at already?  Where are the biggest gaps?
3. Start making small changes and close in on those values.  Share your successes, and keep trying to get better.

It’s lots of work, and there’s every risk of failure.  But it’s better to try and improve your situation than be stuck with something that’s broken.

“Why?” is possibly the worst question you can ask someone when you want to understand the reason for their decisions.

I’m sure you’re perplexed by that statement.  You say, “I already know I shouldn’t take answers at face value.”  My point is that in your quest to find the reasoning behind the answers, you may unintentionally lead yourself astray from the real information you need.

I often have the need to ask “why?” in my work during usability testing, interviews, or getting design feedback.  “Why do you like design A versus design B?”  “Why does pink not work here?”  “How did you know to click on that button?”  Answers to those questions really help me do my job.

Unfortunately, humans are bad at introspection, so the explanations we give are not always valid.  You can usually believe someone’s decision (ex. “I like the blue socks better than the red ones.”). But when you ask for an explanation for why she made that decision, the reason may not be related to this specific incident (ex. “I like blue better than red.” versus “The red is too strong for my white shoes.”).  In fact, believing someone’s explanation could lead you down the wrong path altogether.

What underlies this conundrum?  And what should you use instead of “why?”  I think a little experiment will help you understand the trouble “why” can create.

A little experiment

Let’s say I ask you to take part in my little experiment.  I bring you into a room with two ropes hanging from the ceiling, like below.

“Your task,” I say, “is to tie these two ropes together.  Go.”  Instinctively, you try the naive solution.  You grab one rope and pull it towards the other, but it’s just not long enough for you to grab both at the same time.

You can’t quite figure it out.  I walk by the ropes, accidentally causing it to swing just a little.  A few seconds later, you get an idea to start swinging one of the ropes.  While it’s swinging, you grab the other rope and pull it towards the first one.

When the first one swings back, you catch it and tie the ropes together.

Then I ask you, “how did you know that was the solution?”

Time-out for science

This was a real experiment conducted in the 1930s and reanalyzed in a famous paper “Telling More Than We Can Know” by Nisbett and Wilson published in the 1970s.

There’s only one reason that explains how you got over your mental hurdle to solve the rope puzzle: I nudged the rope just a little.  That means I’m expecting to hear only one answer from you when I ask my question: “I saw the rope swing just a little bit, and that sparked an idea.”

Unfortunately, people in this experiment gave anything but that answer:  “I thought of monkeys swinging through the jungle.”  “I was reminded of a trapeze act I saw at the circus as a kid.”  “When I was in Hawaii, I did a rope swing into the ocean.”  After being pressed, about a third of participants mentioned seeing the rope move.  That means two-thirds of them had no idea what really triggered their answer.

This was one of several experiments that Nisbett and Wilson re-evalutated.  Their conclusion was that we have little or no introspection into our cognitive processes.  Further experiments proved that we have some introspective capabilities but it is very limited.

The impact of our limited introspection

As much as we think of ourselves as rational, self-aware beings, we are not fully aware of our decision making processes.  The extent of this awareness depends on the individual and the circumstances.

To oversimplify, the kinds of answers people give in response include:

• The real answer – It’s possible that you’ll get the true justification for that person’s decision. In the rope swinging experiment, you would hear this as, “I saw the rope swinging, and that triggered an idea.”
• Shared knowledge – A plea to information or theories that are public knowledge or known to the group.  “Everybody knows that you swing a rope, so that’s what I did.”
• Personal experience – The person will use their own beliefs, knowledge, or experiences as their answer.  “I remember swinging on a rope swing as a kid.”  That might have actually happened, but it’s not what triggered his answer.

When someone answers your “why” question, it could be the real answer, it could be something they pulled from their memories, or it could be something else completely different.

And this gets to the heart of the dangers of asking “why.”  The response you get could send you down the wrong track.  For example, take this (fake) usability test snippit:

You: Did you have any trouble filling out the form?

Subject: Yeah, the button was a problem.

You: Why?

Subject: It wasn’t clear what it does.

Actually, the reason why is that the button is blue, and the subject secretly hates blue ever since his third grade soccer team lost the city finals to a team wearing blue jerseys.  But there’s no way to know that from his answer.  His comment sends you on a wild goose chase to redesign that button due to someone’s vendetta against blue.

Similarly, imagine you’re getting feedback about your research into a new product:

You: …and based on our interviews with customers, they want a mobile version of the app.

An executive:  I don’t think that will cut it.

You:  Why?

An executive:  That didn’t work for our competitors, and it won’t work for us.

It’s more likely that the executive has other priorities in mind or that he is biased against mobile apps, but the answer that came to mind first was “it didn’t work for the competition.”

Yes, you shouldn’t take answers at face value, but many people use “why” as a way to cut to the real answer.  The answers that people give to “why” may take yourself off the path towards the real answer due to these limits on our introspection.

If I can’t ask “why,” what should I do?

First, you can usually believe the person’s initial decision.  Imagine you’re choosing two pairs of socks and you immediately like the blue one more than the red one.  You can give any reason you want for liking the blue one; nothing will shake your conviction that it is the superior pair.  Same thing goes for other decisions.  If someone dislikes your design, then they don’t like it.  Accept their decision that your design has problems regardless of the reason.

Next, be careful not to ask justification-seeking questions.  The questions, “why did you click on the games tab?” “how did you choose games next?” and “what drew you to click on games?” are really the same question.  Any answer when asking about a person’s internal state during a decision is suspect by default.

Instead, try asking questions related to the action just taken.  For the person who chooses “games” among several options, the most likely reason is because he likes games.  “Do you like games?  How much do you play them?”  Questions of that sort will quickly prove your assumption and provide valuable information about your subject.

Similarly, if someone doesn’t like a design, you could ask questions like ,”what were you expecting to see?” or “how does this compare with [some other design the person liked]?”  You could even try a more open-ended prompt like, “tell me more about that.”

Also, you can ask the individual to give an example or tell a story instead.  For example if you’re researching video editing software users, you might be tempted to ask, “why do you need video editing software?”  A much better question would be to ask for a specific time when they used it.  “Tell me about the last time you used video editing software.  What prompted you to use it?”

Stay on your toes, and think hard about the information you want to get back from your next question.  You can usually come up with a better question than “why?”

This means something

I find great solace in these results for a few reasons.  First, it justifies the need for psychologists and psychiatrists.  These sciences are concerned with helping people live better lives by understanding our thought processes.  Humans are poor at introspection, so it’s helpful to have someone objectively look at your actions and help you understand how you chose them.

Also, it’s good for your relationships.  Say you’re arguing with your significant other and spout something mean like, “at least I’m not towheaded!”  Then she gets all weepy and says, “why would you say something as horrible as that?”  You can answer, “I don’t know, but I know why I don’t know.”*  Be thoughtful when trying to understand someone’s justification for a decision.  Nobody is fully aware of why you do the things you do.

Finally, this research justifies the work of designers, ethnographers, and similar professionals.  Much of what we do is based on our understanding of the individual(s) we study.  It’s hard work;  asking good questions is just a small part of what we do.  That’s why capable professionals in these fields are worth every cent you pay them.

Take this to heart the next time you start a new project or when you’re interacting with a friend.  Even though humans suck at self-reporting our justifications for decisions, you can still get the information you need without asking “why.”

Coda

For my final project at UC Berkeley, a group of us studied Berkeley freshmen and how they used technology in their lives.  Among the many topics we covered, we asked about their habits using file sharing software.  No surprise — nearly all of them did it.

The next logical question we asked is, “why do you use file sharing software?”  The answers were all over the map.  “I don’t want to help the big music moguls.”  “I don’t have the money to buy it.”  “It’s free!”  Reflecting on that now, “why” was a poor question to ask.

I hope your endeavors in your own research are better for this tidbit of thought.

*Don’t try this in a real argument.  It doesn’t work.

You may think you know your users, but that’s not good enough if you want to design great products for them. *cough Google Buzz cough*

I think this clip from The Simpsons where Homer gives Marge a bowling ball for her birthday epitomizes the problem.

Buying a present for someone is very similar to designing for someone. You’re not that person, so you can only guess what that person might like — a bottle of perfume, a bowling ball, a new man, an awkward social networking system. You often get it wrong, to the anger and disappointment of the recipient.

Why does this happen? Here’s an interesting study that gives us one insight. Researchers brought in couples and separated each pair. One person was asked to rate 30 items of furniture, and the other was asked to rate 30 different items.

Then each person was shown 30 new items — the items their partner just rated. Each partner rated the new items based on either 1) how their partner would rate them or 2) how a stranger would rate them. After rating an item, the person would see the other’s rating, helping them predict future ratings.

How well did each partner predict the other’s preferences? Since these people are couples, you might guess they did pretty well on this test. Actually, it depends on how similar the two partners’ tastes are.

When the two people in the couple have similar tastes (as determined by how they rated the same items), their prediction accuracy was around 40% — whether or not they were in the “rate your partner” or “rate a stranger” group.

When the two people in the couple have different tastes, the “rate a stranger” group was about 23% accurate while the “rate your partner” group scored a whopping 5% accuracy rate.

In other words, if your tastes are different from your partners, you’re better off treating that person like a stranger than trying to guess what he or she really wants. If your tastes are similar, your guesses for the other person will still be so-so.

How does this apply to design?

Most product designers think they know their users. This experiment reminds us that you suck at guessing what they want even when you know them:

• If you know your users, your guesses about their preferences will be awful if your preferences are different than theirs (5% accurate)
• If you’re similar to your users, your guess about their preferences will still suck (40% accuracy)
• In any case, you’re better off imagining your users as strangers (23% and 40%) versus people you know (5% and 40%)
• Getting feedback on your guesses doesn’t help you guess more accurately in the future

Let’s say you’re a large software engineering company based in Mountain View, CA building a new social networking product. You only test it internally, and everyone seems to think it’s ok. Then you unleash it upon the masses, resulting in all kinds of strange backlash that you never imagined.

I say that nobody should be surprised about these results. Even if Google is the golden child of Silicon Valley, they were bound to make this mistake because of their culture.

Why does this happen?

I can think of at least three reasons why people mistakenly think they have the same tastes as their users. These causes are often tied to the culture and processes in a company.

Not accounting for differences

Your users don’t think the same way that you do.  However, because of this nasty thing called projection bias, we mistakenly believe that other people have the same knowledge, ideas, experiences, and feelings as we do.

While that seems pretty obvious as a statement, I’m sure you encounter it all the time in the form of, “it’s great! The users will love it,” or “I know what they’re thinking. They want more social features!” That usually indicates you’re making decisions based on what you think rather than what your users think.

In-house experts

The real killer is when you think that “we’re users too” even though the people at your company aren’t characteristic of your user base. Because of availability bias, we base our decisions on information at hand rather (like our coworkers) instead of seeking out alternative points of view (like our users).

The people at your company aren’t very representative of all users of your product; your coworkers are usually product experts (top 1%) while most users are intermediates or beginners. If you decide on features based on internal feedback, then you’re building for the experts rather than the other 99% of your users. In other words, “we’re users too” leads to products aimed at the wrong audience.

Not asking first

Sometimes you can design something and be right about your user’s preferences.  However, there’s only one way to be sure — ask your users. Because feedback doesn’t improve your accuracy, you have to ask up front — before you lob the new features at your users.

Are you doing any user acceptance testing? With real users who work at other companies and do their work outside your company walls? How do you close the loop with your users after you release new features? It’s the difference between “honey, would you like a bowling ball for your birthday?” versus “honey, I bought you a bowling ball for your birthday.”

Back to the Buzz

So why did Buzz mess up? I hope it’s now clear that Google was playing Russian roulette with its product:

• Google employees are vastly different than their overall user base
• Google designed the product for themselves
• Google only tested Buzz internally before making it public

And presto — Google just gave all their users bowling balls for their birthdays. You get a bowling ball! And you get a bowling ball! You ALL get bowling balls!!!

In summary, knowing your users isn’t enough to build great products for them. If you know them, you can still be horribly wrong at predicting their preferences (like the 5% result). Even in the best case, you’re still in coin-flip territory (the 40% result).

If you want to make better products, start by becoming aware of the biases that affect your decision making.