Who do you trust?

My friend has a new job, and it struck me as very interesting what she is going to be doing. She will be a "content analyst", reviewing product reviews submitted by end users on various websites before they are posted; the her job is to ensure that the reviews discuss the products themselves, and do not make profane or irrelevant statements, or discuss competitors products by name. Ideally, the idea is that the product reviews should be helpful to potential buyers.

I really like the idea of feedback being vetted before it's generally available to users wanting to make an informed decision. I'll give you an example of why. A singer/songwriter I like recently released a new album, and this album is available from a well-known legal music download site. This site allows users to post reviews. There are about 10 reviews of this album, and 2 of them mention nothing about the album; they only talk about the artist's involvement in a recent reality television show. Those reviews are entirely unhelpful, and frankly just make the people who submitted them look stupid. However, there were other reviews that talked about the songs, the artist, and the album. But out of the 10 reviews that are posted, only about half of them are actually helpful; the rest are about as helpful as someone you don't know telling you they like or dislike an album, but without telling you why, and without telling you anything about the song or album in support of their endorsement. Which is exactly what the idea of online reviews is.

So I ask you, if you were in a strange city, would you walk up to a random person on the street and ask, where should I eat? Most of us are more likely to ask people we come into contact with while visiting that city -- people we know who have been to or live in the city, people we are working with, people who work at the hotel where we are staying -- than a complete and total random stranger on the street. Why is that? Because it's nice to know the source of the information you are soliciting. Perhaps the "random stranger" is just as accurate as the person we work with, but there's a "safety factor" that comes into play when soliciting recommendations from people who are known. I have one friend who is a total food snob, and not in a good way. He has no appreciation for burger joints and holes-in-the-wall. He is not someone I would ask where to go for an informal meal with friends, because his perspective is so biased against informal places. But, if I needed a really nice place to go on a date or for a really special occasion, I would ask him. I might not choose a place he recommends, but I would definitely want to know his opinion. I have another friend who does appreciate informal restaurants, but I would never ask him for a recommendation on a restaurant because I know he likes the food at some places that really don't impress me.

So, let's get back to the idea of online ratings. Some people call this type of social computing activity "collective intelligence", but I have also heard it referred to as "collective stupidity". How do you control for Groupthink? And how do you know if the 5 out of 8 that liked that new restaurant you are considering going to for dinner tonight are really one person with 5 screen names? And oh, by the way, that person owns the restaurant? And how do you know that the seller on that auction side from whom you are about to buy a $10,000 item, the seller that has really good feedback from every person she's ever sold a $3 beanie baby to, is really a grifter who has patiently sold low value items and delivered them with the intent of building up her good feedback, but is about to take your money and walk away?

In short, you can never know the agenda of a person posting feedback, so you have to use some common sense. Look at the feedback they've posted before, if possible, before trusting someone's advice on a purchase, major or minor. And use some common sense. If everyone is complaining about battery life, maybe that digital camera really does eat batteries. But if only a few are, maybe there's a reason.

But what about American Idol? You may or may not be aware of Sanjaya, a young man who managed to survive week after week despite repeated lackluster vocals. Is this really because of a campaign by votefortheworst.com to have people call in and vote for him, because these people wanted, essentially, to call into question Idol's credibility? The interesting part about this is that we will never know how many people voted for Sanjaya out of spite, and how many voted for him because they really liked him. On one episode, there was an 11-year old girl who reacted to Sanjaya the way teenagers all over the US reacted to the beatles 40 years ago.

Back to that online auction site....last year, I was going to buy something from a seller I had not bought from previously. It wasn't an expensive item, just a child's toy. I looked at the feedback, because this person had only sold to about 10 people so far. The first piece of feedback I read was very negative. I was a bit curious. The other 9 pieces of feedback were either positive or neutral, and no one complained about anything like excessive delivery time, or not getting what they ordered. But this one piece of negative feedback really stood out. It had profanity in it, and questioned the integrity of the seller. We are talking about feedback on a $10 transaction for a barbie. Out of curiosity, I looked at the feedback this person had left for others. All of it was negative, and the person had said things like "Won't buy from that b_tch again", and used other profanity. Frankly, the reviews were the rantings of a person who was paranoid and insane, and I do not mean either of those figuratively. Obviously, in that case, it was clear that the feedback was untrustworthy, but that's probably an exception.

Again, we come back to the question of whether the person providing feedback, either through digg this or del.icio.us, or through any online rating service, has an agenda, and how you control for that if you do not moderate your feedback. And if you do moderate your feedback, do you leave yourself open to criticism that you are only letting "washed" criticism through?

What's nice about the company my friend is going to work for is that it is a service provider, ostensibly not vested in the material contents of the feedback, only whether or not the feedback meets their criteria: must talk about the actual product, must not talk about competitors, must not contain profanity, and must provide meaningful information; there no stipulation that the feedback must be favorable. Note that my friend will not be editing the reviews, only marking them as "pass" or "fail".

On another occasion, we shall discuss who should be reading, and reacting to, online feedback. And at that time, I'll relate the story of my parents' recent purchase of a minivan. Until then, remember to participate in life; there's nothing visceral about watching something on a TV screen or a computer monitor.