Online reviews not a reliable indicator

Original source: The New York Times

Bart de Langhe, an assistant professor of marketing at Leeds School of Business at the University of Colorado, used to see reviews online and accept them implicitly. Then, when his son was born, he needed to buy a car seat but noticed that the seat rated lowest by Consumer Reports got a high rating on Amazon whereas the one rated highest by Consumer Reports received a low rating.

The more popular seat on Amazon was also more expensive. Were reviewers, he wondered, paying more attention to things like price and brand than the ability of the seat to protect its occupant? With two other researchers, Philip Fernbach and Donald Lichtenstein, Mr. de Langhe began a study that compared online reviews for items like air-conditioners and car batteries with the evaluations in Consumer Reports.

… the consumer saw a number – 4.6 stars out of 5 – and took it much more seriously than it deserved

After analyzing 344,157 Amazon ratings of 1,272 products in 120 product categories, the researchers found “a substantial disconnect” between the objective quality information that online reviews actually convey and the extent to which consumers trust them.

In other words, the consumer saw a number — 4.6 stars out of 5 — and took it much more seriously than it deserved.

Nearly half the time, Amazon reviewers and the Consumer Reports experts disagreed about which item in a random pair was better. Moreover, average user ratings did not predict resale value in the used-product marketplace, another traditional indicator of quality.

Mr. de Langhe concluded: “You should rely much less on reviews than you currently do.”