themeat.org

Great product teams know why people love or hate their products, not if people would recommend them to friends.

NPS – net promoter score – is a metric that many companies use to assess product satisfaction. I’ll assume you know how it works, but if not, see this footnote for a quick primer.

Give me a few minutes of your time to prove to you why NPS sucks and how you can measure love instead.

Why NPS sucks

It’s trademarked
NPS isn’t some commonly accepted measure like temperature or length. It’s a number invented by an agency and it’s been trademarked. If I was an agency and I was looking to drum up work, I’d invent a loaded metric that always shows your product needs improvement, lock it up with intellectual property laws to prevent others from using it, then I’d sell you a solution that optimizes just that metric. Let’s dig into how loaded it is.

Ignoring 28% of the results
I swiped a ton of NPS data from indexnps.com – a site that has lots of NPS results for a wide swath of brands. Recall that responses of 7 and 8 are ignored in NPS results, yet 28% of NPS responses fall in that range. If I was an agency trying to get a client, throwing out a quarter of positive results without any justification is a great way to stack the deck in my favor that you need my help.

Biased results
People often answer survey questions based on what they think you want to hear rather than give an honest response (“experimenter effect“). In the case of NPS, they’ll answer that your product is great even if it isn’t. A full 67% of the indexnps scores were 9-10. Were those people really excited to recommend those products or were they trying to satisfy the pollster?

What are they recommending?
The people answering your NPS question could be recommending any part of your product. For example, if your website has an engaging user experience and separately a monetary incentive for participation, people could be recommending your monetary incentives (“Hey, get free money here”) over your user experience (“Hey, this site is really engaging”). This is an essential detail that gets paved over with the overly simple NPS question.

Hypothetical vs. concrete
Since NPS is framed hypothetically (“Would you recommend…?”), you’re not learning if people actually recommended it (“Did you recommend…?”), to whom they recommended it, how they recommended it, and whether or not that recommendation actually had an effect. NPS leads to more questions than answers.

It’s not an outcome
Companies treat NPS as a key outcome — ex. “Our goal this quarter is to increase NPS 20%” — without understanding how or if that translates to other business results. NPS isn’t an outcome, high scores doesn’t ensure more business, and it doesn’t tell you if customers will stay loyal to your product.

Measuring Love

My hypothesis is that love and hate are the best indicators of your products’ future success, so you need to measure them.

Humans make decisions based on emotions (then justify their emotional decisions with reason). Your goal is to make a product that triggers positive emotional outcomes so that people become loyal to your product without realizing it. Negative emotional responses and decreasing positive emotional outcomes are signs that you’re going to lose customers.

Ask these questions instead to learn whether or not people love your product.

“How do you feel about the product?”
Have the person choose from a series of smileys with a range of emotions – elated, happy, neutral, angry, crying, sick, love, etc. Keep track of the positive versus negative emotions and how they change over time.

“What do you love most about the product?”
Sure, it’s a naive question, but it bluntly gets at the issue. These are the items you should feature in your marketing materials and sales pitches to get future customers’ attention. Ask it open-ended (vs. multiple choice) for best results.

“What do you hate the most?”
These are the problems that you need to fix or you’ll lose customers. Note that different types of users will have different problems, so you need to ask a wide swath of users. For example, if you’re making a business app, the everyday user will hate some parts (ex. “this page sucks”) versus the ones that the buyer hates (ex. “this costs too much”).

“Have you recommended…”
Ask this yes/no question – “Have you recommended [my product] to a [friend/colleague/etc] in [the last month/year/ever]?” Try to learn about the context of the recommendation and why that person recommended it. Concrete questions are way better than hypothetical ones (“Would you recommend…”).

“How did you recommend…”
Ask how someone explained your product to another potential user of your product. Your everyday users can explain your product to other potential users better than you can. You need to understand the language of your users, then use those words when talking to other prospective customers.

Measuring love is hard and necessary

The promise of NPS is that it’s a quick way to assess how well your product is doing — that recommendations are a proxy for product success. Taking shortcuts with something as complex as social interactions or emotional reactions can lead to worse results than doing nothing at all.

This gets to the core of the matter — a great product team should always seek to understand why people love and hate their products, and they should be be able to tell you why without hesitation. “Hey PM, what do people love the most about the product?” If your product team can’t answer that question immediately with concrete examples, they’re not doing their job.

So do the right thing with your NPS scores — get rid of them — and instead learn how people feel about your product. I’d take one person who loves my product over a hundred who’d recommend it to a friend.

—

tl;dr on NPS: NPS is usually asked as a question in a survey like: “From 0 (not at all) to 10 (definitely), how likely are you to recommend [my product] to a [friend/colleague/etc]?” The NPS score itself is (% of people who answered 9 and 10 – % of people who answered 0 to 6).

So if your nps is 70% (that is, there’s a 70% difference between people who answered 9-10 versus 0-6), the general interpretation is that people like your product. A score of -20% (that is, the percent of people who answered 0-6 was 20% more than people who answered 9-10) probably means that people don’t like your product.

“Is it just me or does every product manager feel like this?”

My former, product manager coworker asked me this while walking the streets of Manhattan. He had recently gone through some work shit and was struggling with it — experiencing the doubt that strikes at the heart of anyone in a tough work situation. He asked that question rhetorically, as if he expected the universe to give him an answer so he could resolve his doubts and move on.

I stepped in for the universe. “It’s not just you. We all feel that way.”

Product management is a difficult, burnout-inducing profession. We do it because we love the idea of it more than actually doing it. I’ve hit the point in every product job I’ve ever had where I tell myself, “this is the last product job I’ll ever have.” Still, I keep doing product work, hoping that the next job I have will be the one that fulfills my expectations.

What will it take to close the gap between our expectations and reality in the product management profession?

—

Let’s unpack the job of a PM. We take an ill defined idea and turn it into something that people will love.

That means we need to understand what people need, lead a team to create an amazing solution, ship it into the world, see what happens, and then start the whole process over again. If all goes well, the people who use the product pay us (ideally with money) in return for how much they love the product. The job sounds pretty great.

It sounds pretty great, in theory.

In practice, a product manager struggles to balance the needs of:

• the company who depends on the product’s success
• the customers who pay the bills
• the users (who may or may not be the people who pay)
• executives and others who impose their demands
• new problems which have to be dealt with ASAP
• themselves and their happiness

That last one is the hardest by far. We product people are unhappy. That’s more than just my opinion.

“Product manager” was named one of the unhappiest jobs for two years straight, 2011-2012. Don’t believe it? Ask a PM about his/her job and you’ll see the sadness in their eyes. I can’t remember the last time I met a PM who was truly happy with his or her job.

Why are we product people so unhappy? The explanation is deceptively simple. There’s an enormous gap between the expectations of what we want in a product job and the reality of doing the job.

Let me describe the gap in more detail.

You expect:	ownership of the decisions and direction for a chunk of the product
In reality:	you’re expected to adopt the same decisions and opinions as your boss & managers

You expect:	authority to make decisions based on hard data, user needs, and other evidence
In reality:	HiPPOs (Highest Paid Person’s Opinion) make decisions despite or withholding evidence

You expect:	to work with a close knit team to focus on delivering that product
In reality:	you can’t deliver because the team and priorities change too often

You expect:	enough support to make the product as successful as it can be
In reality:	you capitulate because you don’t have the resources to make it successful

You expect:	responsibility to make the decisions that make the product successful
In reality:	you have to get layers of approval, and your decision can be overturned by anyone above you

You expect:	recognition if things go well, or blame if they don’t
In reality:	no recognition when things go well, and all the blame if they don’t

You expect:	bottom-up management — you tell the execs what they should be paying attention to
In reality:	top-down management — someone else tells you what you’re going to do next

You expect:	a job where you’re the fearless leader of a team who fights to achieve awesomeness
In reality:	you’re a middleman who takes the specs from the customers or managers to the engineers

That gap between reality and expectations is called burnout. The larger the gap, the faster you and your team will descend into underperformance and depression. And that, in a nutshell, why PMs are so unhappy.

Despite all that, we still do the work — hoping that the next job will be the right one where we can do product management “the right way,” where our reality will match our expectations. And if there is a gap, hopefully we’re paid well enough to ignore that gap.

But that’s a horrible capitulation. There’s got to be a better way to do this.

—

What does product want?

We want to…

• make something that people love
• own our customers’ and users’ goals and problems
• make decisions based on data and evidence, not opinion
• have the authority to define our own success
• have the responsibility to achieve our success
• get the resources we need to reach that goal
• have the trust of our stakeholders and managers

That last one — trust — is the hardest one. “Why should anyone trust the product team?” I’ve heard that said too many times and too many ways in my product management career. “First, product has to earn the trust of the organization. Then they can take ownership.”

Therein lies the problem. A healthy team is built on trust — trust that the product team knows how to do their job, how to understand users, how to work with stakeholders, and how to make the right decisions that will make the company and product and users successful.

A healthy team trusts by default.

When trust is lacking, you’ll get the “in reality” situations I described. HiPPOs tell the product team what to do because they don’t trust the product team to make decisions themselves. The company’s course changes constantly because nobody trusts in the product team’s direction. The product manager is blamed for failure even though the product manager never even made a decision by his/herself.

Fixing the problem begins with trust.

To all you executives, product management leaders, or anyone who interacts with your product teams — if you want a healthy product team, you have to trust them with the authority and responsibility to do their jobs. And you should be uncomfortable to cede that much power to them.

It’s a difficult request. Product management is a critical role in a company, and everybody has an opinion on what’s the next most important thing to produce. Giving that power away is a difficult decision, but it’s one that’s essential if you want to see a product team thrive.

The best product team I ever worked on was built on trust. Each PM had a well defined role and knew how their product area laddered up to the company objectives. We were held to goals that we defined for ourselves, and we were given the authority to achieve those goals as we saw fit.

The worst teams I’ve been on, on the other hand, were built on all the opposite. The management questioned and trumped the teams’ decisions regularly. There was no clear ownership and no authority given to the product teams. The team was told what their goals were. There was no trust, and no path to achieve that trust.

If you don’t trust a team or are so uncomfortable that you won’t cede that power to them, fire those people and replace them with people who you do trust, with people who you would empower with real ownership.

Because if you’re not willing to trust them, you don’t have a product team. You have a bunch of overly-paid people who pass specs from the managers to the engineers.

So make the decision to trust your product team. Work with them to understand the problems facing your organization. Help them come up with amazing solutions to those problems. Believe in them when they ship their solutions. And celebrate their wins to show your thanks.

That’s what product really wants — trust. And that will make your product managers happy.

If you’re selling experiences, copies and forgeries will not devalue that experience if your experience is truly worth experiencing.

—

Believe it or not, the following pictures are illegal, and you definitely shouldn’t see them (and I definitely shouldn’t have them).

Many years ago, I saw Spamalot, the Monty Python musical. Immediately after taking the photo of the intermission curtain, an usher rushed over to tell me what I did was against the law and insisted that I delete it right there in front of him. (Obviously I faked deleting them.)

Yes, those crappy, crappy photos were taken illegally. Seriously, cell phones are banned in NYC theaters. But what’s the point of prohibiting photos when it’s the experience of being there that’s valuable?

Here’s another photo of questionable legality, taken at the Museo de Arte de Puerto Rico.

Before I took this photo, the docent caught me with my camera out. He said, very politely, “please no photos of the art here,” and disappeared. A moment later, he peeked his head around the corner. With a smile, he said, “well, you can take that one,” then disappeared again.

It’s as if he secretly wanted me to share the art but had his hands tied, so he let me off the hook just this once. I only wish the photo had come out better to do him proud.

Finally, here’s one of several I took at the David Byrne/St. Vincent tour a while back.

Before the show began, David Byrne came over the PA and said, “we encourage you to share photos and videos of the show because we’re proud of we’ve done.” You can find thousands of videos uploaded to YouTube of their wonderful concerts. It was a fantastic show. Do yourself a favor and check out any of their renditions of Road To Nowhere.

—

It all comes down to ownership and fear.

For the person who “owns” what I’m taking a photo or video of, they’re afraid that my photo or video will devalue the real thing. Spamalot will lose a patron because he saw a video of it on YouTube. The San Juan art museum will get one less visitor because she saw that print in my photo. David Byrne will sell one less ticket because the whole tour is on YouTube.

They’re all unfounded fears.

I record my experiences for a few reasons — to remember what I’ve done and where I’ve been, to brag about what I’ve done, and to share a piece of what I experienced. Nothing can (yet) reproduce the amazing experience of “being there,” and so photos and videos are the next best thing.

It’s not the “real thing,” so why put handcuffs on what gets shared? For example, after the very first performance of Spamalot, there are no more secrets. Word will get out about how good or bad the show is. Video and audio will leak. The public will determine its success by buying tickets. I saw it twice because I loved it the first time and wanted to share the experience with a friend. If my second viewing was via cellphone video instead, it wouldn’t be the same.

So ask yourself what’s really precious. If the experience of being there in person is the valuable part, then it doesn’t matter what people share about it. In fact, sharing can encourage more people to experience it in person.

Coda

A friend asked me for advice. He was wondering who should be allowed to attend the launch of his Awesome New Thing. Specifically, he was concerned that some of the attendees might try to steal his ideas for their own.

I said, “If your ideas are good, they’ll absolutely steal them. The question is — what is it that they can’t reproduce? What makes your experience unique?”

Because if you’re really onto something, the best validation you can receive is when someone shares the experience with someone else or when a competitor imitates you. So why delay the inevitable? Encourage it instead.

*Yes, I’m aware that photo taking can impede memory. Let me enjoy my ignorance.

I prefer to do my research before leaping to outrage.

Of course, I’m referring to the recent revelation that Facebook manipulated users’ news feeds to see if it had any emotional impact on them.

It’s worth asking the question… just what the hell did Facebook experiment with? And why did they do it? And should I really be mad because of it?

If you’re wondering what my credentials are to offer this analysis, there’s not much. I’m an armchair cognitive and social psychologist, so studies like this always pique my interest. So I’ll try to be a bit silly (though not as silly as usual), and I’m certain to skip over lots of relevant details, and hopefully you’ll learn a thing or twenty along the way.

Background

Recently, social sciences are in the midst of a fad studying how emotions and other things spread in social groups. The fancy words for this are “emotional contagion” which reflects how emotions can be shared among people verbally and non-verbally.

I first heard about this from the Framingham Heart Study which showed how obesity and happiness spread through family and social connections. If you read the details, you’ll learn that you’re at an an increased likelihood of gaining weight if your friend, spouse, or family member gains weight. The same study shows happiness spreads through those ties in a similar, “more likely” fashion.

Further studies have shown that yes, emotions are contagious. For example, this effect has been seen in lab settings. Researchers in China studied Weibo posts to show how anger spreads faster and better than happiness or sadness in social networks. If you’re a believer in evolutionary psychology like me, then you’d say that empathy evolved to maintain healthy social groups (read as: mates).

By contrast, some people have argued that simply using Facebook makes us less happy, regardless of what content we’re exposed to. In her book Alone Together, Sherry Turkle argues that replacing rich, face-to-face interactions with computer-mediated communication leads to an illusion of intimacy; the convenience of communication technology actually causes us to be more isolated. (Yeah, this is a slapdash summary. Just read the book.)

There’s some scientific research to support her perspective. A University of Michigan study found that Facebook use predicted a decline in a person’s well being. They polled students throughout the day, asking for their mood and how much they used Facebook since the previous polling. They found that the more someone had used Facebook, the worse their mood would be. Sadly, the press misinterpreted this as “using Facebook makes you unhappy” because they don’t understand the difference between correlation and causation.

We get it. What did Facebook do?

Most experiments to date have looked for correlation between an emotion in a social network and that same emotion in the experimented individual. Facebook sought to answer a variant of the question: if you decrease seeing a certain emotion in a social network, does it result in an increase of the opposite emotion in the individual?

I’ll spare you the brain-numbing experience of reading the paper with this hasty summary. Facebook with help from some folks from Cornell tested two conditions — whether seeing fewer positive posts would lead to increased negative postings, and whether seeing fewer negative posts would lead to increased positive postings. They randomly picked 600k Facebook users from the FB database for the study.

How did they test this? Facebook manipulated the news feeds those users, showing them fewer happier or fewer sadder posts based on words which matched those emotional outcomes. Then they checked those users’ subsequent posts to see how the emotional content changed. The experiment lasted a week.

It worked. People exposed to fewer positive words expressed more negative words in their posts and vice versa as compared to control groups. The researchers noted that this effect was greater when positive posts were hidden than for the group who had negative posts hidden.

Another result is that this showed how emotions can be affected non-verbally, independent of face-to-face contact. Are the emotional outcomes in our own Facebook posts because of content we see in Facebook or do they reflect, for example, emotional outcomes in our face-to-face relationships as filtered through our posts? This seems to demonstrate the former, that Facebook posts alone can affect our emotional state (in our posts).

Also, people in the experiment group were less expressive overall, while people in the control group (who saw greater emotional content in their posts) used more emotional terms in their own posts. The researchers posit that this rebuts the arguments by Turkle and others that using Facebook makes us less intimate with others.

And here’s the part where you get mad

Reason number one to be mad — there was no need to manipulate users’ feeds to prove the point. Instead, they could have run a lab study to see how the decreasing emotional input affects your emotional state. Such a large scale study with all the data manipulation was unnecessary. I wonder if that occurred to them before they started this study.

Number two — the experiment worked. Kinda. In the various experiment conditions, the percent of positive or negative words budged 0.1% at most. That means this change affected the emotion of 1 out of 1000+ words posted on Facebook. That’s not much of a change. Even if it’s a “statistically valid” result, I wouldn’t brag about it. Also, that change doesn’t mean users were happier or sadder. It only proves that they changed their emotional word choice.

Getting angrier, the researchers stretch to shoot down arguments that using Facebook makes you less happy, like those in Turkle’s book or the uMich study I cited earlier. But that’s comparing apples to oranges. Facebook was checking post content days after the experiment went into effect. The Michigan study was checking on the emotional states of people a short time after checking Facebook. Turkle’s book was ethnographic (interviews). You can’t refute one with the other. This only leads to more questions about why these research results are so different, or why they needed to conduct an experiment of this scale to prove the point.

There’s also the question of researcher ethics — specifically that they went forward with a research plan that you intuitively know would make a group less happy as a result. Honestly, I’m not that bothered by an experiment that would make people happier as a result; we could use more of that in the world. Cornell held the experiment at arm’s length, saying that their researchers “did not participate in data collection and did not have access to user data”. Seems rather disingenuous to wash your hands like that when you could be all but certain of the negative results that some in the experiment would face.

So who in Facebook should have stopped this? Nobody. As XKCD perfectly noted, Facebook would have done this research anyway. And they probably have done this research before but never made the results public. If the results of this experiment lead to people staying on the site longer or clicking more ads, then it was worth it to Facebook.

This leads to another point. If you’ve read the paper, you might have noticed that the most interesting results are missing. Did users in the experiment group come back to Facebook more or less often? Did the experiment had a positive or negative effect on Facebook’s bottom line? Were there any long term effects? In other words, was it worth ($$$) it for Facebook? The paper is painfully silent.

Epilogue

This experiment is just a drop in the bucket. We’re the subjects of hundreds of online experiments on a daily basis. Google might be the most notable of the experimenting bunch (for example, testing 40 shades of blue to see which performs best), and other companies have followed Google’s lead hoping this ruthless pursuit of optimization will lead them to similar success. However, few if any of those studies are made public, and even fewer get the attention that Facebook’s study did.

So why did this experiment in particular trigger so much anger? My hunch is that the outrage lies somewhere between “can we really trust Facebook?” and “fear of missing out on what my friends wrote.” It’s the same outrage that underlies big data’s ability to manipulate our lives which danah is all over. We’re quite helpless against the machinations of their algorithms. And thanks to that research I cited, we know that the anger about this issue can travel very quickly via social networks.

Even if Facebook is manipulating your emotions, then so is Google’s search algorithm, that game you just played on your iPhone, and your circle of friends. To have long term effects, you’d need consistent exposure to the same emotional outcome over long periods of time, in the same way you get lung cancer not from one cigarette but from years of smoking. Facebook is just a small part of your total emotional well-being. Emotional reciprocity over long periods of time is a much more important part of that well-being.

That’s why I see no harm — and lots of potential good — by letting people choose to be part tests like this. Why not let people opt-in to a Facebook experiment to prove that you can be demonstrably, permanently happier by manipulating Facebook posts for years? Add a “Make Your Facebook Happier” button. I could do with fewer curmudgeons and negative posts in my news feed. I’m sure lots of others would agree.

Look, I’m sympathetic about what happened here. There’s never a straightforward answer when trying to balance the needs of your audience with the needs of your employer. I’ve had to make many ethical decisions about the products I’ve managed in my time. You make the best choice you can, apologize if you make a mistake, and keep moving. Even if the decision seems significant, it’s usually small potatoes in the end. Time will pass, people will forget.

So what’s really important? The people around you. If you remember only one thing from this rant, make it this. If you want to change yourself — emotionally, physically, anything-ly — the best thing you can do is to change the people around you. Facebook posts, violent video games, and how much porn you watch are small potatoes compared to the company you keep.

As Seth Godin said:

Who you hang out with determines what you dream about and what you collide with.
And the collisions and the dreams lead to your changes.
And the changes are what you become.
Change the outcome by changing your circle.

A coworker came up to me. “I’m sorry for snapping at you yesterday after that meeting.”

“Huh?” I stalled. At that time of the morning, I was not fully conscious.

“I hope you weren’t upset because of it.”

Ah, that. My brain shook off its slumber to dredge up the memory. We were discussing a problem. My answers weren’t good enough, or the questions weren’t clear. It doesn’t matter. The result was — my coworker gave up in a fit of frustration.

“It’s ok. We’re all under a lot of stress right now,” I replied. I was being honest. The incident had barely registered on my radar. Like I said in my response, I’m sure it was due to circumstances and not because my coworker is a short-tempered asshole. (For the record, my coworker is not a short-tempered asshole.)

“I –”

“Hey, got a second?” Another person interrupted us. My apologetic coworker shuffled off.

About a minute later, I realized I had fucked up accepting the apology. And that got me thinking about the nature of apologies and taking responsibility for problems.

—

The subject line of the email read, “Important Information – Unauthorized Intrusion.” It piqued my interest. Either it was elaborate spam or a poorly written subject line that was actually serious.

I opened the email.

IMPORTANT INFORMATION. PLEASE READ IN ITS ENTIRETY.

Dear Patron:

We regret to inform you that on XX XX, 20XX, Company, Co. detected an unauthorized intrusion into its systems.

“Unauthorized intrusion…” You got hacked? Wow. I am soooooo regretful that you got hacked.

Why the hell should I care?

The answer to that question was at the end of paragraph one, in bold, because really important information should always appear at the end of a paragraph in bold. “Your information may have been involved in this incident.”

You can almost see the CEO holding this smelly apology at arm’s length, pinching his nose as he reads it aloud. He buried the lede in an attempt to soften the impact. My personal information and credit card number may have been stolen, and it’s his company’s fault.

That last sentence should have been the email’s entire first paragraph. “Your personal information and credit card number may have been stolen, and it’s our fault.” And how should he have written the subject line?

“We fucked up.”

—

I cringe every time I read something like “I regret to inform you.” Trite apologies are so… trite. “I regret to inform you that your dog is dead.” That’s believable when said by the veterinarian who tried to save my dog’s life, but it doesn’t work if you’re the person who killed my dog. (“I regret to inform you that I ran over your dog.” — see?)

Same thing applies when your website gets hacked. If my info is stolen, then it’s your fault, not the hackers. “Regret” is not an appropriate response. Instead, take the blame — as in, “this was my fault” or “we messed up” or “I broke your guitar when I threw it onto the plane.”

Fall on that sword.

And then apologize. And apologize with a sincere, personal apology. “I’m really sorry.” Not one of those corporate, dismissive apologies: “we apologize for the inconvenience.” “The” inconvenience — not even “your” inconvenience. How can I believe your company’s apology when you can’t admit that “the” inconvenience happened to a person?

When appropriate, offer a remedy. “Here’s a gift basket of smoked meat products and local hard cheeses because we really love you.” Local hard cheeses wouldn’t make me feel better about my wrecked guitar, but might make me feel better if it appeared in my hotel room after you’ve fixed my botched reservation (and after giving me a free upgrade).

What’s the worst thing you can do? Blame others. For example (actual quote here) — “a third-party criminal actor used hacking technologies to access our databases and may have accessed your personal information.” Fail. How about… “Our inadequate security allowed hackers to access the private information which you trusted to us.”

Don’t blame others, and don’t make me feel like I’m the one who’s wrong. “I’m sorry, but there’s nothing I can do for you.” That little act of disowning the blame puts it squarely on my shoulders. There’s plenty that you can do for me, but you choose not to do it. Liars 😛

Get the picture? There’s truth, and there’s truth. One is sincere, and one is not. One is “I did not have sexual relations with that woman,” and one is “I have had sex with women who work for me on this show.”

If you can’t be honest in your apology, then do us all a favor and don’t apologize.*

—

Finally, when someone genuinely apologizes to you, accept the apology with grace.

This person feels like shit about what happened, has no idea how you feel about it, and wants to apologize to make both of you feel better. The worst thing you can do is dismiss this act of humility. That’s where I fucked up with my coworker.

“I’m sorry for snapping at you yesterday after that meeting.”

“Thank you.”

—

* Don’t even get me started on, “we take your security seriously.”

The death of Steve Jobs is a mirror into our own vanity. Designers — take heed.

Steve Jobs blah blah visionary blah blah design blah blah changed computing. It’s true, and I’m not going to rehash the accolades here.

But there’s a flip side to the innovation. I wrote this as a Facebook status following his death:

Steve Jobs in hell is working 16 hour days 7 days a week at a Chinese factory building the new iPhone 666.

It’s funny only because there’s a dose of truth to it. In all the technology we acquire, not just from Apple, much of the cost that went into creating those artifacts is hidden from our eyes unless something brings it to our attention.

Besides dead factory workers, we pay the cost of Apple products everyday. Have you ever gone to the Genius Bar without an appointment? Broke your iPhone and had to buy a new one? Or had your macbook stolen only to have the police shrug?

Some more jokes at dead Steve Jobs’ expense to drive the point home:

• Steve Jobs in hell just had his iPhone stolen, and the police won’t do shit about it.
• Steve Jobs in hell broke his macbook, so he went to the genius bar without an appointment where he’ll wait for his name to be called… forever.
• Steve Jobs in hell broke his phone but can’t afford a new iPhone, so he had to settle for a free Nokia.
• Steve Jobs in hell got a $5,000 cell phone bill this month because he’s roaming on his data plan.

The jokes practically write themselves. Let’s face it — Apple products are expensive and fragile. They are time sinks and pickpocket bait. And we suck it up because the products are so tantalizing — because owning an Apple product is a statement about you.

If you think I’m denigrating the name of Steve Jobs or Apple, I am, but only as much as it takes to make my point. If it makes you feel better, you could play Mad Libs with the text above and swap out the names of the companies, products, and executives; it would be just as true but the products wouldn’t look as good. Simply put, we pay more than money for the privilege of owning these magical amenities.

Steve Jobs was responsible for Apple’s meteoric rise these last decades because there was an audience desperate for Apple’s pretty products — me included. If there’s any lesson to take from Steve’s passing other than a life cut off in the midst of greatness, it’s that we greatly value the veneer of our possessions despite the costs.

You might think your team does “agile development.”  You’re not agile.

At Infocamp Berkeley this weekend, I ran a session called “You’re not agile.” I fully admit it was inspired by my anger from organizations who claim they’re “agile” when they’re really not.  Here’s an alternate take on what I presented…

A long time ago, waterfall development was the process used to create software.  Executives would make requirements, designers would make designs with the requirements, developers would build software from the designs, QA would test the software, the software would ship, and the cycle repeat.  Software development was treated like factory work because that was the only model of production they had.  Replace “software” with “bicycle” in that explanation and you’ll see what I mean.

However, waterfall isn’t ideal for software.  Requirements change, customers have limited input, and software simply isn’t hardware.  Using waterfall for software is like cooking a souffle using directions for grilling a steak — worst souffle ever.  Similarly, using a process that works great for manufacturing physical goods is a bad idea for producing software goods.

What is agile development?

As a backlash to waterfall, teams began experimenting with new methods.  These groups started with the premise that there must be a better way to produce software in a corporate environment.

For example, waterfall process is obsessed with documentation as the interface between steps of the process.  However, producing functional software is a far more important business goal, even if you skip writing everything down.  Similarly processes and tools are useful, but you can solve problems with communication way faster than following a checklist.

These values and principles are codified in the Agile Manifesto — a document well worth your time to read and understand.  It was written by a group of agile practitioners looking to forge a group identity for their new vision of software development.

Sadly, it lacks a concise definition of what “agile development” means, so here’s my take:

Agile development is a set of values and principles for improving your development teams’ alignment with your organization’s needs.

That’s a dense sentence.  Read it a few more times to make sure you get it.

Take notice that my definition and the Agile Manifesto do not say anything about specific practices that will make you agile.  Rather, agility is a shared mindset that you and your team use to solve development obstacles.  A team with this as a motivating goal is a powerful force for creating kick-ass-ness in the world.

The cargo cult of agile

News of successful organizational transformation with agile development methods started spreading.  Success breeds imitators, and other teams tried agile for themselves.  They started doing agile practices — like daily standups, iterations, and product backlogs — but never gained the benefits.

They became the cargo cults of agile.

The term “cargo cult” refers to a group who imitates another group’s processes but lacks the substance which makes those processes work.  The term was inspired by the actions of Pacific island inhabitants in World War II.  U.S. and Japanese troops would set up bases on these islands.  With these bases came regular shipments of goods, visits from planes, and formation marches by troops.

After World War II, the troops, planes, and bases left too, leaving the native inhabitants without their regular deliveries of goods.  To bring back those deliveries, some of them recreated the artifacts and actions of the visitors — building airports with dirt roads, airplanes and radios from bamboo and coconuts, and marching in formation as best as they could.

Sadly, the natives were unsuccessful in making additonal cargo drops reappear because they didn’t understand the visitors’ airplanes, airports, cargo, and radios.

Similarly, many organizations put agile methods into practice without adopting agile values.  For example, a manager orders the engineering team to start doing daily standups. The engineers perceive this as “managers are spying on what we’re doing” and treat it accordingly.  By contrast, an agile team knows that standups are for the team itself — and not for the managers — because the team values communication, helping each other, and self-organizing.

Because the cargo cult agile teams don’t internalize the changes their making, they don’t get better.  As a result, there’s been a backlash against agile development from the people who’ve had bad experiences where “we tried to do agile, but it just didn’t work.”  They give agile development a bad name.

Lessons learned

In my session, I went over a few agile practices and how they represent the values and principles of agile development, but I’ll leave it up to you to find additional resources on how to be agile.  Instead, I want to touch on a few motifs that ran through the conversations and questions I heard at Infocamp:

• Agile UX — How does design fit into agile?  In short, agile UX means “stop drawing pictures and just start building.”  Organizations are very uncomfortable with this; many UX designers don’t have coding skills to build prototypes, and managers and executives are wary about committing development resources when designs are incomplete.  It will take a real sea change to make this catch on.
• Becoming agile — How do you begin agile development in your organization?  It takes lots of things — a willing development team, executive buy-in, strong values, reasonable tools, and a great agile leader to guide the team.  But what’s even more important is this — start small.  Do one project “agile.”  Remove organizational impediments one by one.  Bring in new practices little by little.  And one more thing — don’t be afraid of failure; just keep trying.
• Management skills — Most schools don’t teach agile.  Hell, most schools don’t teach management skills at all.  The people I know who do agile well learned it themselves or from someone who kicks ass at agile.  In other words, agile development needs better marketing and education to spread the word and knowledge.

Another way to paraphrase the questions is this — “How can I learn more about agile and apply it to my organization?”  I’ll give you three next steps:

1. Learn about agile development.  Read blogs and books about it.  Find people who know about it and pick their brains.  Go see a kick ass agile team and how they work.
2. Decide what you value.  Then start looking at how your team runs. What are you good at already?  Where are the biggest gaps?
3. Start making small changes and close in on those values.  Share your successes, and keep trying to get better.

It’s lots of work, and there’s every risk of failure.  But it’s better to try and improve your situation than be stuck with something that’s broken.

“Why?” is possibly the worst question you can ask someone when you want to understand the reason for their decisions.

I’m sure you’re perplexed by that statement.  You say, “I already know I shouldn’t take answers at face value.”  My point is that in your quest to find the reasoning behind the answers, you may unintentionally lead yourself astray from the real information you need.

I often have the need to ask “why?” in my work during usability testing, interviews, or getting design feedback.  “Why do you like design A versus design B?”  “Why does pink not work here?”  “How did you know to click on that button?”  Answers to those questions really help me do my job.

Unfortunately, humans are bad at introspection, so the explanations we give are not always valid.  You can usually believe someone’s decision (ex. “I like the blue socks better than the red ones.”). But when you ask for an explanation for why she made that decision, the reason may not be related to this specific incident (ex. “I like blue better than red.” versus “The red is too strong for my white shoes.”).  In fact, believing someone’s explanation could lead you down the wrong path altogether.

What underlies this conundrum?  And what should you use instead of “why?”  I think a little experiment will help you understand the trouble “why” can create.

A little experiment

Let’s say I ask you to take part in my little experiment.  I bring you into a room with two ropes hanging from the ceiling, like below.

“Your task,” I say, “is to tie these two ropes together.  Go.”  Instinctively, you try the naive solution.  You grab one rope and pull it towards the other, but it’s just not long enough for you to grab both at the same time.

You can’t quite figure it out.  I walk by the ropes, accidentally causing it to swing just a little.  A few seconds later, you get an idea to start swinging one of the ropes.  While it’s swinging, you grab the other rope and pull it towards the first one.

When the first one swings back, you catch it and tie the ropes together.

Then I ask you, “how did you know that was the solution?”

Time-out for science

This was a real experiment conducted in the 1930s and reanalyzed in a famous paper “Telling More Than We Can Know” by Nisbett and Wilson published in the 1970s.

There’s only one reason that explains how you got over your mental hurdle to solve the rope puzzle: I nudged the rope just a little.  That means I’m expecting to hear only one answer from you when I ask my question: “I saw the rope swing just a little bit, and that sparked an idea.”

Unfortunately, people in this experiment gave anything but that answer:  “I thought of monkeys swinging through the jungle.”  “I was reminded of a trapeze act I saw at the circus as a kid.”  “When I was in Hawaii, I did a rope swing into the ocean.”  After being pressed, about a third of participants mentioned seeing the rope move.  That means two-thirds of them had no idea what really triggered their answer.

This was one of several experiments that Nisbett and Wilson re-evalutated.  Their conclusion was that we have little or no introspection into our cognitive processes.  Further experiments proved that we have some introspective capabilities but it is very limited.

The impact of our limited introspection

As much as we think of ourselves as rational, self-aware beings, we are not fully aware of our decision making processes.  The extent of this awareness depends on the individual and the circumstances.

To oversimplify, the kinds of answers people give in response include:

• The real answer – It’s possible that you’ll get the true justification for that person’s decision. In the rope swinging experiment, you would hear this as, “I saw the rope swinging, and that triggered an idea.”
• Shared knowledge – A plea to information or theories that are public knowledge or known to the group.  “Everybody knows that you swing a rope, so that’s what I did.”
• Personal experience – The person will use their own beliefs, knowledge, or experiences as their answer.  “I remember swinging on a rope swing as a kid.”  That might have actually happened, but it’s not what triggered his answer.

When someone answers your “why” question, it could be the real answer, it could be something they pulled from their memories, or it could be something else completely different.

And this gets to the heart of the dangers of asking “why.”  The response you get could send you down the wrong track.  For example, take this (fake) usability test snippit:

You: Did you have any trouble filling out the form?

Subject: Yeah, the button was a problem.

You: Why?

Subject: It wasn’t clear what it does.

Actually, the reason why is that the button is blue, and the subject secretly hates blue ever since his third grade soccer team lost the city finals to a team wearing blue jerseys.  But there’s no way to know that from his answer.  His comment sends you on a wild goose chase to redesign that button due to someone’s vendetta against blue.

Similarly, imagine you’re getting feedback about your research into a new product:

You: …and based on our interviews with customers, they want a mobile version of the app.

An executive:  I don’t think that will cut it.

You:  Why?

An executive:  That didn’t work for our competitors, and it won’t work for us.

It’s more likely that the executive has other priorities in mind or that he is biased against mobile apps, but the answer that came to mind first was “it didn’t work for the competition.”

Yes, you shouldn’t take answers at face value, but many people use “why” as a way to cut to the real answer.  The answers that people give to “why” may take yourself off the path towards the real answer due to these limits on our introspection.

If I can’t ask “why,” what should I do?

First, you can usually believe the person’s initial decision.  Imagine you’re choosing two pairs of socks and you immediately like the blue one more than the red one.  You can give any reason you want for liking the blue one; nothing will shake your conviction that it is the superior pair.  Same thing goes for other decisions.  If someone dislikes your design, then they don’t like it.  Accept their decision that your design has problems regardless of the reason.

Next, be careful not to ask justification-seeking questions.  The questions, “why did you click on the games tab?” “how did you choose games next?” and “what drew you to click on games?” are really the same question.  Any answer when asking about a person’s internal state during a decision is suspect by default.

Instead, try asking questions related to the action just taken.  For the person who chooses “games” among several options, the most likely reason is because he likes games.  “Do you like games?  How much do you play them?”  Questions of that sort will quickly prove your assumption and provide valuable information about your subject.

Similarly, if someone doesn’t like a design, you could ask questions like ,”what were you expecting to see?” or “how does this compare with [some other design the person liked]?”  You could even try a more open-ended prompt like, “tell me more about that.”

Also, you can ask the individual to give an example or tell a story instead.  For example if you’re researching video editing software users, you might be tempted to ask, “why do you need video editing software?”  A much better question would be to ask for a specific time when they used it.  “Tell me about the last time you used video editing software.  What prompted you to use it?”

Stay on your toes, and think hard about the information you want to get back from your next question.  You can usually come up with a better question than “why?”

This means something

I find great solace in these results for a few reasons.  First, it justifies the need for psychologists and psychiatrists.  These sciences are concerned with helping people live better lives by understanding our thought processes.  Humans are poor at introspection, so it’s helpful to have someone objectively look at your actions and help you understand how you chose them.

Also, it’s good for your relationships.  Say you’re arguing with your significant other and spout something mean like, “at least I’m not towheaded!”  Then she gets all weepy and says, “why would you say something as horrible as that?”  You can answer, “I don’t know, but I know why I don’t know.”*  Be thoughtful when trying to understand someone’s justification for a decision.  Nobody is fully aware of why you do the things you do.

Finally, this research justifies the work of designers, ethnographers, and similar professionals.  Much of what we do is based on our understanding of the individual(s) we study.  It’s hard work;  asking good questions is just a small part of what we do.  That’s why capable professionals in these fields are worth every cent you pay them.

Take this to heart the next time you start a new project or when you’re interacting with a friend.  Even though humans suck at self-reporting our justifications for decisions, you can still get the information you need without asking “why.”

Coda

For my final project at UC Berkeley, a group of us studied Berkeley freshmen and how they used technology in their lives.  Among the many topics we covered, we asked about their habits using file sharing software.  No surprise — nearly all of them did it.

The next logical question we asked is, “why do you use file sharing software?”  The answers were all over the map.  “I don’t want to help the big music moguls.”  “I don’t have the money to buy it.”  “It’s free!”  Reflecting on that now, “why” was a poor question to ask.

I hope your endeavors in your own research are better for this tidbit of thought.

*Don’t try this in a real argument.  It doesn’t work.

You may think you know your users, but that’s not good enough if you want to design great products for them. *cough Google Buzz cough*

I think this clip from The Simpsons where Homer gives Marge a bowling ball for her birthday epitomizes the problem.

Buying a present for someone is very similar to designing for someone. You’re not that person, so you can only guess what that person might like — a bottle of perfume, a bowling ball, a new man, an awkward social networking system. You often get it wrong, to the anger and disappointment of the recipient.

Why does this happen? Here’s an interesting study that gives us one insight. Researchers brought in couples and separated each pair. One person was asked to rate 30 items of furniture, and the other was asked to rate 30 different items.

Then each person was shown 30 new items — the items their partner just rated. Each partner rated the new items based on either 1) how their partner would rate them or 2) how a stranger would rate them. After rating an item, the person would see the other’s rating, helping them predict future ratings.

How well did each partner predict the other’s preferences? Since these people are couples, you might guess they did pretty well on this test. Actually, it depends on how similar the two partners’ tastes are.

When the two people in the couple have similar tastes (as determined by how they rated the same items), their prediction accuracy was around 40% — whether or not they were in the “rate your partner” or “rate a stranger” group.

When the two people in the couple have different tastes, the “rate a stranger” group was about 23% accurate while the “rate your partner” group scored a whopping 5% accuracy rate.

In other words, if your tastes are different from your partners, you’re better off treating that person like a stranger than trying to guess what he or she really wants. If your tastes are similar, your guesses for the other person will still be so-so.

How does this apply to design?

Most product designers think they know their users. This experiment reminds us that you suck at guessing what they want even when you know them:

• If you know your users, your guesses about their preferences will be awful if your preferences are different than theirs (5% accurate)
• If you’re similar to your users, your guess about their preferences will still suck (40% accuracy)
• In any case, you’re better off imagining your users as strangers (23% and 40%) versus people you know (5% and 40%)
• Getting feedback on your guesses doesn’t help you guess more accurately in the future

Let’s say you’re a large software engineering company based in Mountain View, CA building a new social networking product. You only test it internally, and everyone seems to think it’s ok. Then you unleash it upon the masses, resulting in all kinds of strange backlash that you never imagined.

I say that nobody should be surprised about these results. Even if Google is the golden child of Silicon Valley, they were bound to make this mistake because of their culture.

Why does this happen?

I can think of at least three reasons why people mistakenly think they have the same tastes as their users. These causes are often tied to the culture and processes in a company.

Not accounting for differences

Your users don’t think the same way that you do.  However, because of this nasty thing called projection bias, we mistakenly believe that other people have the same knowledge, ideas, experiences, and feelings as we do.

While that seems pretty obvious as a statement, I’m sure you encounter it all the time in the form of, “it’s great! The users will love it,” or “I know what they’re thinking. They want more social features!” That usually indicates you’re making decisions based on what you think rather than what your users think.

In-house experts

The real killer is when you think that “we’re users too” even though the people at your company aren’t characteristic of your user base. Because of availability bias, we base our decisions on information at hand rather (like our coworkers) instead of seeking out alternative points of view (like our users).

The people at your company aren’t very representative of all users of your product; your coworkers are usually product experts (top 1%) while most users are intermediates or beginners. If you decide on features based on internal feedback, then you’re building for the experts rather than the other 99% of your users. In other words, “we’re users too” leads to products aimed at the wrong audience.

Not asking first

Sometimes you can design something and be right about your user’s preferences.  However, there’s only one way to be sure — ask your users. Because feedback doesn’t improve your accuracy, you have to ask up front — before you lob the new features at your users.

Are you doing any user acceptance testing? With real users who work at other companies and do their work outside your company walls? How do you close the loop with your users after you release new features? It’s the difference between “honey, would you like a bowling ball for your birthday?” versus “honey, I bought you a bowling ball for your birthday.”

Back to the Buzz

So why did Buzz mess up? I hope it’s now clear that Google was playing Russian roulette with its product:

• Google employees are vastly different than their overall user base
• Google designed the product for themselves
• Google only tested Buzz internally before making it public

And presto — Google just gave all their users bowling balls for their birthdays. You get a bowling ball! And you get a bowling ball! You ALL get bowling balls!!!

In summary, knowing your users isn’t enough to build great products for them. If you know them, you can still be horribly wrong at predicting their preferences (like the 5% result). Even in the best case, you’re still in coin-flip territory (the 40% result).

If you want to make better products, start by becoming aware of the biases that affect your decision making.

Being in jury duty is like forcing twelve people to order a three topping, extra-large pizza where the three toppings must be different and everyone has to agree on all three toppings before you can order.

Sometime during the course of the proceedings, I realized that the drama in front of me was the manifestation of everything I knew about cognitive science and social psychology. The lawyers are scientists and the courtroom is a laboratory. The lawyers use witnesses and evidence to mix up a verdict through persuasion, rhetoric, and storytelling.

I decided to put on my ethnographer’s hat and record my observations — the action in the courtroom, the drama in the deliberation room, and everything else relevant to life as a juror. Below is a scattershot list of my interpretations.

It’s long and poorly organized, and I’m not apologizing for it. And I’m being intentionally vague about gender and case details.

Backseat lawyers

As a jury, we were extremely dissatisfied with the lawyers for how they argued the case. Following a particularly awful exchange by one lawyer, a juror said he/she wanted to jump out of the jury box and yell at that lawyer for being such an idiot. (That’s almost a direct quote.)

As we went forward in our deliberations, we uncovered essential evidence that both lawyers missed. We often complained that the lawyers could have made a more compelling case by asking questions about X or grilling witness Y.

During one complaining session of that sort, I wanted to get the jury back on track. I said, “let’s hold off on the backseat lawyering.” Someone quipped in response, “Monday morning lawyer.” Exactly.

On a side note, I think a courtroom drama with an MST3K-style commentary track has the potential for extreme humor.

Humor

In case my previous attempts at jokes weren’t funny, we tried hard to maintain our sanity through humor during the deliberation process. This often came out as jokes about the witnesses, lawyers, the court personnel, or even ourselves.

Yes, we were a bit disgusted with ourselves given the seriousness of the courtroom events. I think it was necessary to crack a joke every so often; for me, it kept me from taking the deliberations too seriously — avoid making it personal.

One time while we were commenting on the courtroom antics, I said, “this would make a great screenplay — all twelve of us deliberating in the room. Co-starring the bailiff as our comic relief. And with special guest stars the judge and court clerk.” It was much funnier in the deliberation room; you just had to be there.

Someone said that jury deliberations would make a great reality TV show. “You could film a different jury deliberating every week.” I definitely agree.

I finally understand how someone got a cell phone in my name

About a decade ago, someone fraudulently opened a cell phone account in my name with a major wireless provider. That person racked up hundreds of dollars in calls, then the phone was shut off for non-payment. This showed up on my credit report when I went to get my own cell phone, resulting in countless hours lost as I tried to ressurrect my good credit.

Thanks to the expert witness testimony of a cell phone company employee at this trial, I now know why it’s so easy to do that. Unlike banks or credit card companies, cell phone companies do not extend credit to individuals; you’re bound by a contract to pay for the service every month. Because they don’t extend credit, those companies are not required to check the ID or verify any of the information (like address) that a person provides when getting a cell phone contract.

In St. Louis, a Sprint store employee gave a cell phone to someone who used my name and SSN along with a fake address (“Wadaba St” — seriously, that alone should be a red flag) when I was living a thousand miles away. And Sprint is under no obligation to verify that it was really me (or in this case, not me) who did it. And due to some arcane laws, I would have to go to St. Louis in person to press charges.

Words cannot express my outrage.

Technology (or lack thereof)

Our courts are technologically inept. Most of the evidence could have been trivially scanned and made available to the jury digitally if they had wanted.

Instead, we got 100 pages of phone call records, 50 records a page. (Oh, how we would have loved to build a pivot table.) We had to request special equipment to view a DVD. Every photo had to be passed around to each member of the jury, wasting countless hours. Testimony had to be printed (for some witnesses, hundreds of pages long) or, if the written transcripts weren’t ready, read aloud by the court reporter.

We opined about how much faster we could get through the information if we could get everything digitally. Instead, the prosecution lawyer pushed a cart full of boxes and binders, each full of pages and pages of paper, to court every day. Those poor trees…

The pool

The jury selection process is awful. It starts with 24 jurors being selected randomly from a pool of over 100. Then the judge and lawyers begin asking a series of questions that you’re expected to remember whether or not you were selected. Some jurors are booted, then a new set are called up to fill in the empty seats. More questions are asked, and the process repeats until both lawyers decide to keep the 12 jurors in the box.

It was the most boring three days of my life. Sure, there was the occasional interesting character or curious story. I spent my time devising ways that the courts could accelerate the process and get a suitable jury faster and make it more fun.

But no — the boring ways produce a fair jury of the defendant’s peers. I think the (long, boring) pooling process is the primary reason why people hate jury duty.

The negative opinion of jury duty

Here are some reasons why people dislike jury duty:

• You spend three days in a jury pool but never get picked to be on the case — total waste of time
• You end up on a trial and have to put your life on hold or have to work 20 hour days (8-9 hrs at court + your real job)
• Sitting in the courtroom is boring. Everything moves as slow as possible
• Deliberations take a long time and are very mentally taxing
• You get paid almost nothing ($15 a day plus $2.50 for transportation in SF)

But I think the main reason why people dislike jury duty is this:

• They’ve never been on a jury before

When I told people that I was on jury duty, most people said, “aw, that’s too bad,” or something like that. The only people who said good things were ones who had been on a jury before.

It’s really not that bad. Give it a try sometime.

It’s no [your courtroom/police drama of choice]

It’s no CSI. It’s no Law and Order. In the courtroom, a real trial is much more boring than you think. Lots of questions and answers. No magic technology to reassemble events of the incident. No outbursts by witnesses. No need to restrain the defendant.

I guess courts shouldn’t be exciting or dramatic, given the gravity of events that occur there. On the other hand, they shouldn’t be boring either, given the gravity of events that occur there. I’ll think about ways to improve that.

What the witnesses saw or didn’t see

Because the events in this trial occurred over a year ago, lots of witnesses had trouble remembering the events of the incident. In several cases, the lawyers had to remind the witnesses about statements they made to police on the night of the event; at court, the witnesses often misremembered or didn’t recall details.

Even more interesting — the key eyewitnesses (the defendant and a victim) recalled completely different versions of the incriminating event. It’s a form of motivated recall — that is, you’re motivated to remember events that portray yourself in a positive way or in a way that makes you seem better. In this case, the victim wanted a conviction, and the defendant wanted to be released; they testified as such.

As a jury, we were left with the tough decision to decide, individually, whether or not we believed the witnesses in part or in whole.

Everyone needs something a little bit different

As we were deliberating, we discussed the points that brought us to a reasonable doubt or brought us to believe the defendant was guilty. While it was good to air our thoughts, it did little to convince people one way or the other.

The reason why that doesn’t work is because of projection bias — that quirky cognitive bias that leads you to believe (incorrectly) that others share your beliefs or thoughts. The evidence that convinces you of your verdict probably won’t convince another juror of your verdict; in fact, more than once during our deliberations, someone would bring up evidence that obviously pointed to guilt while another juror would use that exact same evidence to point to innocence.

Every juror needed something a little bit different to get to a conclusion — a timeline of events, a read-back of certain testimony, an alternate explanation for a witness’s observation. If there’s any lesson from the trial that I’ll apply to my life, this is it — it may be perfect to you, but it does nothing for others (whatever “it” is — evidence, love, designs, hamburgers, Charlie Sheen movies).

The hardest part of the deliberations, by far, was finding that “little bit different” for each juror. How do you convince people of your opinion? I still don’t know the answer to that. All I know is that we deliberated for 7 days and didn’t agree on all the charges.

(Speaking of projection bias, in deliberations a few jurors said, “I would have…” or “If I was there…” If the witness did something that you wouldn’t do, the witness must be lying, right? Don’t assume others think or behave the same as you.)

The sounds and the smells

The Hall of Justice at 850 Bryant is smelly because it’s the home of the San Francisco parole and probation offices. Every day there would be interesting odors while waiting in line for the security check, going up the elevator, or, most interesting of all, in the stairwell.

The odors included pot, crack, alcohol, and smelly homelessness. Often I would see the security guards put on rubber gloves when dealing with homeless people. The security guards also kept a wooden walking cane near the x-ray machines to be used when a homeless person’s bags got stuck in the x-ray machine.  (They would use their hands on others’ belongings.)

And about the stairwell… it reeked of pot. Every day, without fail. My only wish is that the parole and probation officers have noses, because I bet lots of the people who show up are breaking their terms of probation — as determined solely by the odor.

The grammar

Having been through this trial, I’m much more grateful for my education. I’ve now heard the phrase “I seen” more times than I ever want to in a lifetime.

Court lingo

You learn lots of new terminology while in court. Words you thought you knew like “robbery,” “murder,” and “intent” have specific definitions while you’re there. Here are some favorites:

• Dixied: to steal. See also: boosted
• 6 pack: a photo lineup used by police to identify suspects
• Green vegetable matter: marijuana, when entered into evidence and not yet identified as weed
• Stipulate: an agreement between the prosecution and defense about the facts of the case
• Intent: I’m still not sure what that means

Reinventing the wheel

Even though jury trials take place all over the country, most jurors are inexperienced and perform jury duty infrequently (less than once a year). That means juries are reinventing the process for deliberating over and over all the time. In my jury, nobody knew the right way to run the deliberations, so we made it up as we went along.

Said differently, everyone is a beginner in a jury. My user experience instincts tell me that you need plenty of hand-holding as a beginner in a complex process that you go through infrequently. Courts should help juries as much as possible to get through the deliberation process. Instructions would be great.

Instead, we were left with “1) deliberate. 2) vote. 3) if unanimous, tell the judge. else goto 1.” If we had some rules or advice for how to deliberate, I think our task would have been completed much faster. A different presiding juror (foreman) might have shortened (or lengthened) our deliberations by a day; I want to know what process juries can use to shorten deliberations by days.

The instructions

At the end of the closing statements, the judge gives the jury instructions for how to decide the case. For example, “robbery” has a very specific meaning including dispossessing someone of an item, using force or fear to deprive the person of the object, and intending to keep it permanently.

In our case, the instructions were over 50 pages of the dry, difficult legal jargon. As non-lawyer people, we could only decide among ourselves or ask the judge what they mean. And because it’s over 50 pages worth of dense stuff, there’s no way to keep all that in mind when deliberating.

The definition of “robbery” included a 5 step process with special rules and exceptions, and it refers to other definitions that have their own steps and processes. The criteria for believing or not believing witnesses was 5 pages by itself. Circular references were common. One juror joked that the instructions should be in a wiki.

The output is only as good as the input; simpler instructions would help juries come to better decisions faster.

The missing pages

Even though you’re not supposed to assume or concoct ideas, you can’t help but do that, especially when you find the witnesses’ testimony to be dubious. We tried to reconstruct what really happened, sometimes taking liberties with the evidence we were given.

In a more amusing moment, someone suggested that a victim inflicted his wounds on himself.  I said, “you mean this is like Fight Club?” It got a laugh.

As humans, we have an innate drive to finish stories — a drive for completion. Trying to reassemble the real story of a court case is like reading a novel with every other page torn out or watching a movie in a language you don’t understand. You invent your own story; you have to.

Small potatoes

During our casual chats, we often talked about how the case was impacting our lives. Several people reported that they had trouble sleeping, including one who thought a person was standing in his/her bedroom (it was a shadow). A couple of folks said they started noticing suspicious characters in their neighborhood. A couple of people were at the trial during the day then left to work a full time job at night.

As the trial went on, the impact started getting more severe. One person said his/her manager was getting annoyed about when that juror would get back to work, including veiled threats about his/her employment status. Someone had a family member undergo surgery during the trial. Another had a close friend with a death in the family. Another a dying relative.

Someone suggested we ask the judge for a day off to tend to our lives. One of the jurors put it into perspective. “Whatever issues we have in our lives that are affected by this trial, it’s small potatoes compared to what the defendant, the defendant’s family and friends, and the victims’ families and friends have gone through. As much as we have a duty to our lives, we also have a duty to see this case to its end for those people.”  Small potatoes indeed.

Reinterpreting everyday life

Someone said they noticed a strange car driving by his/her house and got suspicious. Another juror said that he/she was being more argumentative in everyday conversations. Someone else mentioned that he/she is trying to be more observant during activities in case that moment ever became the subject of a trial.

Another juror said that the process had made him/her more thankful for life and the privileges therein. I definitely agree; many countries don’t have jury trials or the disposition of innocence that we have in the USA.

I think we were all transformed by this process, and I think my comments above only scratch the surface of the impact it had on our lives.

Closing Statement

It wasn’t easy being a juror, but I do feel satisfied with the process, the quality of my fellow jurors, and the results of our effort.

I feel like I need a mental vacation to recharge myself.

And I have tons of other observations about the trial that I haven’t even gotten to yet.

I might need another blog post to tie this one off…