Too many tech companies put their efforts into search and queries but not enough time in evaluation and discovery.
This is one of those topics that has weighed on my mind a lot, brought to the front by what I wrote last time. I claim that search has four main parts:
- Discover
- Query
- Execute
- Evaluate
To run through the steps in order, first you have to discover the thing you want to search for. More on this further down.
Next, you have to formulate that discovery into a query. With that query, you can execute your search in any way you see fit — asking friends, using search engines, flipping through magazines, sending messages by carrier pigeons, and so on.
After you’re done executing the search, you can evaluate the search results to see if they match your need. If so, great — you’re all done. If not, hop back a few steps and try again.
My rant on Yelp was mostly about evaluation; Yelp’s results did not match my query. And really, Yelp never will be able to answer my query because it’s not built to answer that kind of query.
My concern is that most companies don’t put effort into Discovery and Evaluation. Most companies focus on the Query and Execute parts of the search process. Let’s face it — technology lends itself best to those steps.
Evaluation is key; its the sin qua non of what a search engine produces. Bad search results mean nobody will use your product. We humans are the true judges of the effectiveness of a search.
But how many of those companies check if their search produces great results? For example, I know that Google hires people to evaluate search results and make sure they’re sane to humans. Based on my results, I doubt Yelp does the same (or maybe Yelp does it very badly — hard to tell).
Get on with it
If Execute and Query are industries and Evaluation is languishing, then Discovery is on the back of the milk carton. Think about Discovery like this:
How did I first discover X and decide that I wanted learn more about it?
You’re exposed to many facts and tidbits throughout a day that you mostly forget or discard. Just a few items stick around in your brain, like that song you heard while watching TV and you know you heard it before but you don’t know the name… What is that song?
Maybe you remember one line of the lyrics —
If a girl has a pierced tongue, she’ll probably suck your dick
So you punch it in to a search engine and start your search, quickly adding quotes after you get too many porn sites back.
"No Sex in the Champagne Room" by Chris Rock. And elation — you solved the mystery, all triggered by your discovery of the song and remembering one lewd lyric. Then you went and downloaded the MP3 of the song. QED.
Paths to Discovery
Discovery comes in many forms. Some are pretty square — education, reading Ain’t It Cool News, walking down the street on a windy night when a gust blows a newspaper right into your face open to an article about a kickin’ rock show happening tomorrow night…
Here are a few of my favorites:
Recommendations
Sometimes discoveries come to you. For instance, my friends and family often tell me about restaurants I need to try. Sometimes, I even go to them.
Recommendations are powerful constructs as I previously attested. The only missing bit from last time is that they often induce you to start a search process.
Seeking Out
Sometimes you’ll go out of your way to find information. For example, I actively look for news on The Cinematic Orchestra — tours, new albums, bootlegs. I love the band.
You might also call this one "fandom" or "love." It’s so powerful that it will drive an individual to seek out that information all by himself. This is the most powerful discovery mechanism — self-motivated, active searching — compared to other methods that are more passive or induced externally.
Stumble Upon
No, not the website. This is a catch-all for the methods of discovery like hearing a catchy song in a bar and wanting to learn what it is. Or maybe you saw an ad for an upcoming movie while watching a TV show and want to see who’s in it.
The key to stumbling upon something is that it comes in unexpected times and ways. This is the realm of marketing and advertising — filling in the gaps of time and space with information that you’ll follow up on. Often, their goal is to influence a decision like the billboard that unconsciously determined your laundry detergent purchase.
Sometimes you’ll be motivated to do more with something you stumble upon — like hearing a song at a pool hall in St. Louis which led me to buy my first Bela Fleck album, or hearing a song on the radio which led me to buy my first Cinematic Orchestra album.
Good things come in unexpected ways.
Search to Discovery
Sometimes your searches will lead you to new paths of discovery. I’m reminded of Last.fm and Pandora that fancy themselves "discovery engines." They work on the premise that they’ll find music you’ll like based on music you already like.
But really, these sites are big search engines using criteria like other people’s music preferences (Last.fm) or an expert opinion (Pandora) to help you discover new music.
All the same, I’ve certainly found good music through those services. Or maybe you watched a YouTube video then clicked on one of the recommended videos after the video was over. Or maybe you bought the thing that Amazon said other people purchased when they bought the first item. Your search led you to a new discovery.
Lessons from Discovery
There’s good news and bad news.
The good news is that technology is catching on to discovery. I’m reminded of that iPhone app that can listen to a segment of a song then take you to the iTunes store to buy it. That answers the search need right at the time and place of discovery.
And like I said, Pandora, Last.fm, and their ilk help turn search into discovery. Or YouTube introducing you to related videos while you’re watching one.
The bad news is that there’s definitely room for improvement. Even those examples above are really search examples — searching for music like the band you like, or finding videos similar to the one you’re watching.
Is this the best way technology can facilitate discovery — through search? Have databases become so ubiquitous that technology will only be good for search and querying?
Netflix friends is on the right track, but it’s hard to motivate people to put in their recommendations, much less for everyone they know. Reviewing is not the answer; as I said before it leads to poor answers for useless questions. (Just look at the state of the Netflix recommendation project.) Anyone who can solve this recommendation conundrum — without resorting to search — is bound to make millions.
We could enable better stumble upon discovery if only car commercials put a blurb in the bottom corner of who wrote the damn song. I know lots of them are written just for the commercial, but at least give the artist props so I can seek them out.
But I digress. Everyone is looking for something new — a new restaurant, a new band, a new movie, a new event — and the only answers that technology offers are directed search and recommendations pulled from the sky (which is really another form of search).
Get your search right and you might discover something interesting. Get it wrong, and we’re all gonna laugh at you.
Technology has few answers for the serendipitous discovery — the song in the TV ad or the friend who thinks you would love this movie. These are the kinds of discoveries that broaden our horizons. My life didn’t change that much when the latest Cinematic Orchestra album came out, but you should ask me about the first time I listened to The Cancer Conspiracy…
Or maybe you met that special someone when you tagged along with a friend to a party and she also tagged along with a friend to that same party. These kinds of moments do not happen because of a great search engine. I’m sure eHarmony and others would like you to believe otherwise.
Is there a way that technology can facilitate those moments? I say yes, there is a way that technology can make this better, easier, faster than before. I’m just not sure what it is…