Update – the fake reviews featured below were removed shortly after this post blew up on Hacker News.
One reason we buy from Amazon: plenty of reviews. But what if many of Amazon’s top-reviewed items have fake, paid reviews?
I was looking for a sunrise alarm clock this morning and started searching through the many reviews, filtering by ones that mentioned “minutes,” since I wanted to learn about the product’s timer feature. This surfaced a bunch of similar-looking reviews:
Here, we see both the top and bottom review with the sentence,
The light can be pretty bright, you can adjust it where it’ll be dim and slowly brighten 30 minutes before the alarm time.
Did “Becky” and “Dione Milton” really both happen to write a review with the exact same 23-word sentence? Or, is it more likely that they are agents sourcing reviews from a script, and they sloppily pasted their reviews without rewriting them (as they were presumably instructed to do)? Note also the post dates: December 12, 2017. “Becky” and “Dione Milton” both had private profiles, where their 5-6 reviews were hidden – very similar looking.
Amazon – who has some of the world’s most advanced ML – really needs to step up its review fraud detection game. Imagine how great the Amazon shopping experience would be if we could trust its reviews.
Third party meta review sites like Fakespot will identify problems for us (in this case, the product got an “F” grade) – so why doesn’t Amazon?
Amazon: you can do better.
Update 2020-09-23: you might want to also watch “why Amazon has a fake review problem“
30 replies on “Amazon’s Fake Review Problem”
The reviews read “verified purchase”. You think they actually order a few dozen items just so they can write a verified purchase review?
Absolutely – it’s economical, given the importance and dominance of the Amazon marketplace. What’s a better alternative explanation for these reviews with duplicated sentences?
Not to give it all away but…here’s a hint.
Amazon now faces the same problem that Google faced several years ago thanks to blackhat SEO. Pagerank was vulnerable in the same way that Amazon reviews are now vulnerable.
Google found the solution, and the solution is the same for Amazon’s issue. It could be implemented by a small team in a long weekend using only the data they already have.
I can see how the blackhat analogy holds, but am skeptical that a small team could solve this over a long weekend. Maybe Amazon needs to hire you? 🙂
And what’s “the” solution?
Ehm… Yes? Specially if the store or manufacturer is the one behind it.
If I want to review my own product, I will buy it to show ordered, and then resell on same Amazon. All i have to do is spend some money on shipping at most? Verify purchased doesn’t mean that guy even open the box.
I agree and Amazon should take action on this. I’m pretty sure they already notice this problem but what about sales? Maybe in their mindset they prefer to allow some fake reviews in order to maintain current sale volume. Just a thought.
Perhaps there are short term (e.g. quarterly) sales targets within Amazon that lead to some perverse incentives (bad for the long-term)
“Amazon calculates a product’s star ratings using a machine learned model instead of a raw data average. The machine learned model takes into account factors including: the age of a review, helpfulness votes by customers and whether the reviews are from verified purchases.”
This makes sense. It just doesn’t seem to be working too well so far, if the cited model is in fact active.
+1
Try using fakespot.com or reviewmeta.com. They’re both sites setup to try & check the validity of Amazon comments using different methodology – such as how many times a reviewer has left a review, certain trigger keywords known to be used by bots & how many reviews Amazon has deleted from a review by request of the merchant (yes, Amazon will do this!). This will leave you with a much better sense of whether or not a product has been ‘review-bombed’ or if the company is up to shady practices.
Pretty happy with these sites so far, and will continue to test and recommend them. Good suggestion!
Fakespot etc are themselves scams. They are anonymously registered (hidden identity) Adsense sites and you don’t know who they are or what’s under the hood.
Not sure how you’re going to “test” them — you would have to launch your own products & get reviews you know are real (by interacting with your customers) — that are then all labeled fake — to know these sites are fooling you.
So, very clever meta-scams, easy to fall for, as you’ve done here. They don’t really work, but as long as they appear to work, it doesn’t matter. People will continue to promote them, and they’ll continue to make a fortune in ad sales.
Not sure if you’re just trolling here. You count the anonymous registration as a strike against them, yet you yourself appear to have provided me with a fake email address so I cannot contact you.
Admittedly, getting ground truth data (known valid or invalid reviews) is difficult, but we can come up with signals that are clear evidence of fraud. Having a site like Fakespot examine this evidence for us is a much easier first-pass than manually weeding through thousands (or more) data points.
I contacted someone at Fakespot, who in his thorough reply to me, appears to have good intentions. He has a background in a relevant industry that would make him very resourceful in this space.
I only read bad reviews anyways….
Like the post above, I also concentrate on the bad reviews. Helpful on ebay too where sellers hide their location in China and shipping takes weeks.
I too typically focus on negative reviews. At some point, unfortunately, this incentivizes fake negative reviews from competition.
Amazon only does something when the public spot light is on and not enough. Take a look at this Chinese seller and his products. All the reviews 5 star unverified on the same exact random day in November. I reported to Amazon and they have done absolutely nothing. These sellers in China have a boat load of disposable seller accounts if they get caught, shill buyer accounts leaving reviews, direct links to the factories selling
Junk, and they don’t pay a dime in US state, federal, or sales tax.
https://www.amazon.com/Vertical-Multi-function-Controller-Playstation-Dualshock-4/product-reviews/B0776XYJ85/ref=cm_cr_getr_d_paging_btm_3?ie=UTF8&reviewerType=all_reviews
Wow, that’s another pretty flagrant case of review fraud.
Interesting. Fakespot (mentioned in an earlier comment) corroborates your view: https://www.fakespot.com/product/ps4-vertical-stand-cooling-fan-xuzou-multi-function-fan-cooling-stand-charger-controller-charging-station-with-dual-charger-dual-usb-hub-charger-ports-for-for-sony-playstation-4-dualshock-black
If Amazon starts deleting blackhat reviews there will be legitimate reviews mistakenly flagged. Probably very few, but it only takes one story going viral to be a huge PR issue, and if the review is deleted how can anyone tell whether Amazon was right to do so? People are very sensitive about claimed abuses by large corporations, even when there’s no way to verify them.
Not to mention that any sophisticated technical approach will launch an arms race with the blackhats using Amazon’s catalog as the battleground.
Stop the world? The problem needs to be addressed. I’ve personally had a product that I’ve sold for years in the marketplace badly damaged by competitor sabotage very obvious fake 1-star reviews. You have no idea who frustrating it is to see your livelihood attacked and have no recourse.
Truth is that amazon reviews NEED a barrier to entry. I don’t see any way to address the problem other than to require that every person have only one account, and that a credit card and physical address be attached to every account.
The marketplace is too large for efficient oversight, and algorithms are never going to be omniscient. If the algorithm improves, there will be a corresponding improvement in the quality of fake reviews. As it stands, most fake reviews are sloppy and easy to spot. But who knows? Maybe there are 5 undectable reviews for every ridiculous, sloppy and obvious ones.
Unintended consequences are a bitch. The last time amazon “fixed” the problem, it made incentivize reviews a TOS violation. I think most people discountd them. Now, fake reviews are verified. My dirtbag competitor used to buy them, and they didn’t seem to move the needle. Now, he is almost certainly buying “real”verified reviews.
Another thing I would like to see? Strip away the anonymity. Why not require peopl to own reviews that they leave. No more “Amazon Customer.” Make it “John Smith, Thousand Oaks, Ca.” Would have no problem with that if I liked a product and wanted to elate an honest review. Likewise, if I were cheated or sold a faulty item, I would be happy to amend m actual name and identity to an honest 1-star review.
Sure, it would reduce the number of reviews, but the corresponding increase in trustworthiness would give honest reviews much more currncy, and offer honest sellers a layer of protection.
It’s great to see stories like this. I do t think most people are aware of the level of fraud in Amazon’s review system. The more people lose faith in them, the better. A loss of faith and an impact on the bottom line is the only thing that will motivate Amazon o invest the massive resources needed to even begin to address the problem.
As for the verified purchase, it’s a technique called “brushing”. You make real purchases so you can leave fake ratings. Maybe you even return the items.
eBay has a similar problem where a new seller needs to get a lot of positive feedback. They start by selling very cheap things (say, $0.99 phone chargers) to pump up the number of transactions and 5 start ratings. After that they have a seller account that is valuable and can transfer it or do something else.
But this technique isn’t new. You can mess with the New York Times bestseller lists by buying your own book at NYT reporting stores. You make small purchases to fly under the corporate purchase radar.
You’ll never solve this problem though. It’s an arms race and the other side has much more incentive to game it. Neither does Amazon or eBay have incentives to change it. Purchasers get used to it but keep buying stuff. The people who get annoyed are the makers of rival products and they aren’t going to stop selling either.
I wasn’t aware of the extent of the eBay problem, and never heard it articulated before – yet as a teenager, I intuitively realized this and went so far as to sell some cheap items to build up my account reputation before selling a “large” item so it’d fetch a higher value. Couldn’t eBay simply weight the lower-dollar items less – but it sounds like perhaps they don’t?
Seems to be a very difficult marketplace for the honest seller today.
Something I have noticed many times, is large numbers of 4 and 5 star reviews for the wrong item. Sometimes the actual item that was reviewed is similar, which makes it harder to notice. Other times, it was quite obvious. Verified purchase seems pointless here, if the review of the purchased item isn’t the item under which the review is posted.
This happens a lot when the seller lists multiple items (that aren’t even related) under the same umbrella listing. You know how you can sometimes select various options for an item? Sometimes, the seller makes those options completely different items. :facepalm:
There are exceptions, (as in the one posted above), but why shouldn’t paid reviews be seen as a form of marketing for some sellers. Truth is, if you’re a smaller seller in an over populated product category, your listing will often not be given a respectful amount of traffic, due to the A9 not acknowledging your listing. One of the few ways around this is to have a 5 star review and a specific amount of verified reviews. Especially when going up against established sellers with hundreds of reviews. I don’t have the answers, but I’d be curious to find out how some of you propose to balance the playing field that Amazon has set up.