The Grand Illusion
The other day my wife made the observation that almost everything seemed to be “rated” these days. Rate your stay at the hotel or motel. Rate your purchase. Rate the service and food at the restaurant. Rate this book. Rate this movie. Rate your car. Rate the teacher. Rate the doctor. Rate the professor.
When I was in college, too many years ago, about the only things that were rated were a handful of very high-end restaurants… and they were rated by anonymous experts. Now, almost everyone can rate almost anything. But for all those ratings… have matters changed all that much? Even as millions have rushed to rate, exactly how much do those ratings mean? And is their effect more in what is bought or sold or more in boosting the companies offering the ratings? In the case of Amazon, the ratings definitely boosted sales, and probably affect to some degree what is bought, but, as I’ve discussed before, the ratings certainly don’t measure excellence, only popularity. As for other companies in other fields, the results are at best mixed.
There’s definitely an effect in areas where millions pile on, so to speak, if only because the amount of ratings suggest a certain popular appeal… but, again, that doesn’t reflect excellence necessarily, just popularity, a fact that’s particularly overlooked in such spectacles as “American Idol” or “America’s Got Talent.”
What also tends to get overlooked is that the more things are rated, the less respect there is for the area being rated. The idea of rating Einstein on a scale of one to ten, or one to five, seems ludicrous now, but how long before we get to the point of “Rate the Scientists”?
Even at Amazon, the ratings game can be absurd. How does one make a meaningful comparison between Pride and Prejudice and Pride and Prejudice and Zombies without disrespecting the original? And some comparative “ratings” clearly point out the absurdities, as when Americans give Congress approval ratings of something like 18% while a majority of voters in most congressional districts approve of what their representative has done.
And, once more, as I’ve pointed out, the idea of 18 year olds having any idea of what they’re doing in rating college professors is absurd. They’re “excellent” in picking popular teachers, but the only meaningful correlation is that the professors with the highest student evaluations, in 90% of the cases, are those who give the highest grades… not the ones who demand the most of their students.
So… on a scale of one to five, I’d give most ratings a negative grade, not that what I say will do a damned thing to change or even slow the ratings madness.
I think the problem lies very much in the “stars” or number rating. I can give a professor 5 stars, but how is that useful? It doesn’t convey any meaning as to why I believe he/she deserves such a ranking – which I would argue as far more important. Any rating without some substance behind it explaining why a particular rating was given is rarely going to be useful.
The ratings race is getting very annoying – especially since most ratings can be gamed, making them even more useless.
I just returned my printer back to where I bought it. The wireless capability quit and so did the scanner. Even though the model is not made now, it was still under warranty. I’m glad I paid $10 for an extra year. Since the new model that replaces it was on sale this week, now was the right time to replace it.
At the end, the cashier showed me the website on the receipt where I could rate the service that I received this morning. Normally I forget about all that before I leave the store. I find the rate this and rate that mania to be beneath my notice.
This time I’m going to make an exception. I’m going to rate my service because it was excellent. Oh, did I mention that I supposedly will be entered into a draw for a $1,000 shopping spree at the store if I do so? I really could use a new desk and chair. So I’ll play along. This time.
I don’t care what most people think about an item or service. Most of what I need is simple – food, clothing, etc. and those I can figure out with my own eyes and my own experiences. When I go to choose a doctor, I don’t ask the 350 people on the street what they think about Dr. X, I ask the people who work with them “Would you let her take care of you or your family?” I ask the car salesman what car he owns. I ask the real estate agent where THEY live. Experts or people with a lot of experience in their own field have credible opinions. When it comes to noteworthy decisions for my family and myself, I don’t listen to an unexperienced rube’s ‘gut feeling.’ I’d rather listen to the expert or experienced person’s ‘gut feeling.’
Notice I don’t say ‘considered opinion.’ Most times, people make decisions like this using their gut and come up with facts and reasons later. 8 times out of 10, I’m no different… I just don’t like lying to myself about it.
Its the “who is judging the judges” type race, but I think both Amazon and Google use the weighting obtained from other people clicking “Had this review been helpful to you” Yes/No, and adjust the weightings of of all ratings by the user, based on that plus/minus score.
That’s why after I buy something from Amazon based on reviews; after evaluating the product, I go back and click Yes/No on the reviews that were spot-on, and click “No” on bad faith ones.
Public ratings mostly reveal the expectations of the least competent and most vociferous members of the audience, rather than a competent evaluation of the product.
Private ratings can however be useful. Netflix uses them to tell you what people with similar tastes liked. This works surprisingly well.
I rarely take ratings into consideration when buying something or going to a place like a restaurant or service-related business, because I’ve found that many times people only rate to complain about something. The majority of people who bought the item or ate at the restaurant had a good experience and didn’t bother rating it.
Rating instructors and things like that seem pointless to me. I get surveys at work to rate my boss or the “all hands” meetings and I usually skip them because they are too generic and not worth the time, and nothing ever comes of them that I see.
I do like to read through what other people say about things like movies, books, albums, etc, though, expert or not. And as Joe says the ratings systems that recommend other items based on your rating history are nice to have, I’ve found some good stuff I wouldn’t have otherwise because of that.
I had a software supplier once who wanted me to rate their service. The first question was “did we perform better than expectation?”. On a scale of 1-5 I responded with a 3.
The CEO contacted me to ask why so low a score. So I explained that the reason they secured the contract was that I had high expectations and they met them.
If I had awarded them a 5, then I would have had low expectations.
They did not get this at all.