Google, Yelp and Why Review Counts are Meaningless

Filed in Featured, Google, Local Search, MY BEST POSTS by Matt McGee on July 5, 2011

google-yelp-logosGoogle has been copying Yelp’s approach to “social local” for some time now, with things like its Hotpot ratings/reviews service, the creation of “community managers” that work on the ground in several cities to increase awareness and usage of Google Places and several other things that I detailed on Search Engine Land a little while back.

But there’s one important area where Google is doing exactly the opposite of Yelp, and it’ll probably be the measure by which experts say Google has someday surpassed Yelp. What’s Google doing so differently than Yelp?

In one sentence: Google is incentivizing reviews at almost spam-like levels. Consider a recent event that Google hosted in Portland:

On June 6th, Google and a local arcade called Ground Kontrol announced a “special party” with free “food, drink and prizes provided by Google Places.”

This is important: The event was scheduled for June 22 — just 16 days after it was announced.

So, how did you get an invitation to this special party? Here’s how the announcement explains it:

The first 100 people who have at least 50 reviews of businesses on Places by 5pm PST on June 19th will get an invitation to our Game On! party for a free night of arcade games, with food and drink on Google Places.

In other words: If you review 50 businesses within 13 days, you get an invite to the party! You basically have to spam Google Places with reviews to meet that requirement, unless you’re some kind of Herculean local business shopper/visitor/reviewer!

But wait, there’s more! If you managed to go beyond 50 reviews in less than two weeks, you got extra prizes:

* 50+ reviews will grant you unlimited game play, appetizers, and 1 drink on Google.
* 100+ reviews will grant you unlimited game play, appetizers, up to 2 drinks on Google and a t-shirt.
* 200+ reviews will grant you unlimited game play, appetizers, up to 2 drinks on Google, a t-shirt, and a $15 gift certificate to Ground Kontrol (for future use).

Did it work? Well, Google wrote about the event on its own blog afterward, and said that 10 people reached the 200-review mark and got those extra bonuses listed above.

WOW.

This Wouldn’t Fly on Yelp

Yelp takes a 180-degree opposite approach to reviews. Yelp lives in what I’ve called “a world of fairies and pixie dust” because it doesn’t even want local businesses to ask for or encourage reviews … and they have a specific policy against incentivizing reviews.

Yelp holds local events like the one Google did, Yelp has community reps and all that — heck, Yelp pretty much wrote the playbook on this kind of real world outreach. But Yelp, to my knowledge, doesn’t pre-condition attendance at its parties based on users posting a certain amount of reviews, nor does it give users prizes based on how many reviews they write.

Result? Review Counts are Meaningless

What’s the big deal? Well, aside from the possible pollution of Google Places with hundreds of Portland-area reviews that are less than stellar due to people trying to write 50+ reviews in a couple weeks … probably not much. And, to be fair, I have no idea what the quality of those reviews was; for all I know, those 10 people each wrote 200 excellent reviews. (And I suppose it’s also possible that some/all of the ten already had written 200+ reviews on Google Places.)

But what it does mean, though, is that, when Google starts bragging about how big its review corpus is and people start proclaiming Google the king of local reviews, we need to

  • take into account the tactics that Google used to surpass Yelp, and
  • ask ourselves if review counts really mean anything

Other than a nice headline now and then, I’m betting they won’t. Not when Google is giving free food, drinks and prizes to the users who can crank out the most reviews in short periods of time.

Comments (16)

Trackback URL | Comments RSS Feed

  1. mkb says:

    Yelp incentivizes reviews and other database cleanup by granting Yelp Elite status, which gets its members into exclusive events. Also, everyone gets a free sweatshirt.

  2. MiriamEllis says:

    Great reportage, Matt, and I feel myself wanting to wag a finger a Google for hypocrisy. If Google can offer all kinds of swag for reviews, why can’t my clients? Hmm? Would you like to answer that, Google???

  3. mkb says:

    It’s not hypocrisy. Google isn’t attempting to influence the content of a review, only its presence. If your clients are attempting to pay for good reviews, that’s a different ball of wax.

    (I meant above that Yelp Elite members get free sweatshirts, not every member of site)

  4. Rocky says:

    Yes, Yelp incentivizes. But not anywhere near the degree to which Google has been.

    I lived in Portland during much of this and Google was over the top with these incentives.

    Note that they also have restaurants encourage people to write reviews. See this Google Places checkholder:

    Yelp discourages businesses from soliciting reviews.
    http://www.flickr.com/photos/asmythie/5807464742/in/set-72157626087313732

  5. GregB says:

    Wow, now I don’t feel so bad about trading tee-shirts for reviews.

  6. Kathy Long says:

    Good point, Matt.

    Lest we forget though, while they may be meaningless when it comes to rank, reviews are important for searchers trying to find the best restaurant or car mechanic. Do you need 75? No. But enough for people to weed out the wheat from the spam chaff.

    Sure wish Yelp would clean up its act. They are sOOOOO guilty of weeding out the good, legitimate reviews.

    • Matt McGee says:

      Thanks for the comments, everyone. 🙂

      I wouldn’t call it hypocrisy, Miriam. Google rewrote its review guidelines months ago and narrowed down what they think is fine and what’s not. Contests and such are perfectly fine in the current guidelines. Your client certainly can offer swag for reviews on Google, but Yelp would frown on that.

      mkb – thx for the info.

      Kathy – I’m actually not talking at all about search rankings here. My point was just about the race that’s bound to happen over which site has the most reviews. Google will eventually surpass Yelp and be declared the king of reviews, but it will have done so via tactics like what I’m describing above.

  7. Jim Rudnick says:

    @Matt…nice one, lad! didn’t know anything about the Google ‘review-for-invites’ thingy….but yeah that’s not a solid way to gain my own trust, eh!

    🙁

    Jim

  8. Barry says:

    This goes to show that although GOogle uses review count as a signal to local rankings, it certainly can’t use it as a big signal…..especially given the fact they’re manipulating their own results.

  9. Nyagoslav says:

    @Matt – I don’t think they are doing this for the sole purpose of incentivizing mass spam reviews on Google Places. I wouldn’t think that 2000 or 3000 reviews more over 2 weeks period are going to make a big change in terms of “review corpus”. It’s more of a branding tactic and that’s how they are using the community teams in my view. A good proof of that could be the “A Google A Day” initiative. They are getting their product known among the locals, so I don’t see anything bad in that. Of course things like “prize for 200+ reviews” are a little overexaggerated, but the core idea is definitely not spamming.

    @Barry – I am not 100% sure if the reviews by Google Users are a ranking factor, or if they are – they are a real minor one IMHO.

  10. Matt McGee says:

    Nyagoslav – I’m not speaking to Google’s purposes and motivations. I’ve no doubt that their goal is to build awareness and increase usage in the community. If I gave an impression to the contrary, I apologize.

    What I’m writing about above is the industry aspect – the battle between Google and Yelp to own the review space. Yelp dominates that right now. Google is doing everything it can to change that.

    And yes, a small burst of reviews in one city for one event isn’t much in the grand scheme of things. Until you have Google doing events exactly like this 3-4 times per month in 25 different cities across the country. Believe me … that’s not too far off in the future.

    All I’m saying is this (again, apologies to all who seem to be missing my point): When Google is declared the leader/winner in the review space, let’s remember how they got there.

  11. cham_001 says:

    The protagonist’s view: “… good ‘old-fashioned’ on-the-ground marketing techniques – roll-on Google for ingenuity!
    the antagonist’s view: “… shocking! I tell you… what dishonest and downright dirty tactics! – may Yelp reign supreme on quality!
    My view: regardless, Google has set the tone – we are spending valuable time debating it!
    Conclusion: Google has won this battle and will no doubt eventually win this war regarding the Review-space.

  12. Chloe says:

    I think you’re also mistaking Yelp and Google’s types of reviews. Yelp reviews are usually super-lengthy 4 or 5 paragraph affairs, while Google’s tend to be 2-3 sentences. Doing 50 2-3 sentence receives in 13 days doesn’t seem complex at all.

  13. Cody Baird says:

    I dislike anything or anyone that muddies the waters or misleads the end users experience. Its even more frustrating when I hear of Google doing it. I wish there was a better way or process to assure that people are leaving reviews for the correct reasons.