User ratings and reviews have been an online fixture for over a decade and a critical part of the operating models of many sites: Yelp and Epinions are built entirely on such content, while Amazon and eBay rely on ratings to support their primary business.  Third-party automotive sites have also buttressed their inventory, research data, and expert reviews with user-generated content.

Manufacturers, on the other hand, have largely resisted putting anything resembling user content on their sites (as described in a prior posting, Social Media on OEM Sites).  Some link to channels on other sites while others have incorporated content from sources such as YouTube and Flickr (still carefully controlled) into a separate section.  Baby steps, for sure.

Recently, Infiniti jumped in feet first by incorporating owner ratings into its model pages – prime screen real estate.  As shown below, the G37 Sedan receives 4.6 out of 5.0, and site visitors can even click to view the individual owner reviews.

The layout follows conventions and will make intuitive sense to anyone who has ever used online ratings.  Infiniti provides filters and sorting functionality, which is especially helpful given that the reviews encompass multiple model years and trim levels.

As with any kind of user-generated content, transparency is critical – it’s only valuable if visitors know there’s no manipulation.  Unfortunately, Infiniti.com gives no such assurances.  There’s no explanation of the source and no “submit review” functionality.   The end result feels like a set of hand-picked testimonials, which diminishes the implied authenticity.

However, I was surprised when I compared Infiniti’s ratings to those on Edmunds (which operates on a 10-point scale).  In only one case (the G37 Convertible) was the Infiniti.com score higher.

The content of the Infiniti.com reviews is generally, but not absolutely, positive.  For instance, some G37 Sedan reviews complain of road noise, the transmission, and/or interior build quality issues, which mirror the concerns of Edmunds.com G37 Sedan reviewers.

Even so, I can’t help but wonder about the source of all these reviews.  Among all third-party sites, Edmunds has the most extensive set of online reviews and doesn’t manage better than 24 reviews among these models.  But Infiniti.com has far more reviews for most of its models, including over 300 for the G37 Sedan!  The content is also uniformly well-written and clearly edited (e.g. proper capitalization, no misspellings, reasonably good grammar), which is not the norm for user reviews.

No, it’s not perfect.  But it’s still a brave first step.

There are numerous online sources that provide vehicle reviews and ratings, both expert and user-generated. In order to test the differences between rating sources, I conducted an analysis based on models in six highly competitive segments. Only those 58 models with ratings across all rating sources were included so that the results are directly comparable.

The sources:

• Cars.com: Consumer ratings
• ConsumerGuide: Expert ratings
• Consumer Reports (obtained via MSN Autos): Expert ratings based on physical testing, drive tests, and qualitative judgment
• J.D. Power and Associates: Paper-based surveys of vehicle buyers regarding their experiences in the first 90 days of ownership
• Edmunds.com: Expert and Consumer ratings
• MSN Autos: Consumer ratings

One clear pattern is that consumer-based sources give higher scores in general. The first grouping below consists of expert ratings while the second grouping is user-based rating sources.

Biases are to be expected with consumer feedback. Survey respondents tend to avoid giving very low scores, so the average on a 10-point scale will generally above 5, the supposed “average” rating. Additionally, self-selecting polls and reviews (like consumer ratings) are not representative of the entire target population, so the consumer ratings are unlikely to portray the “average” owner experience.

Further, the people who submit user reviews are more likely to be either enthusiastic about or extremely dissatisfied. There’s less in-between color that would produce a bell-shaped spread of scores that reflect the actual experiences of vehicles owners. Survey-based research, like that conducted by J.D. Power and Associates, generally represents a broad cross-section of experiences.

The top- and bottom-rated models also varied across sites.

Note that the BMW X5 is a top performer for both cars.com and MSN Autos user ratings, while the Cadillac SRX does well on ConsumerGuide and Edmunds. There are also divergences: the Subaru Impreza does well on cars.com and poorly with J.D. Power and Associates, while the Chrysler Aspen does well on cars.com and MSN Autos user ratings, but poorly on Consumer Reports.

Regardless of the source, reviews and ratings continue to provide a valuable complement to other online automotive research.

Content Targeting – Akamai
Panelists: Brian Apley, Akamai Technologies; David Harris, Suzuki; Bob Doppel, JWT

• Content targeting is the customization of content on the page based on where they came from and what their “online profile” looks like.
• Continually asking the consumer what will serve them is a must.
• Benefits of segmentation targeting:
  • Ford has used geo-targeting to surface the information consumers are looking for (such as dealer info, incentives).
  • Consumers are much more satisfied with the personalization of information presented.
• Some often-found issues with the process of content targeting:
  • Lots of analysis and work to look at the data, come up with segmentation; perhaps the most difficult part of the process.
  • Developing creative that targets each segment individually.
  • Pages to put that creative on – customization on the fly as you’re testing
  • Knowing what to track; more analytics on the page than content!
• Courage to just jump in and do it is the first step. It’s easy to get held up in the process and analytics; everything is fluid, so we always have to watch and listen and adapt quickly.
• Commitment to the testing cycle and iterations is a must.
• The web site is going to stop being the destination and become just part of the journey. Detach the content from the delivery and only giving what the consumer is interested in will be the only way to really reach them in an effective, efficient way.
• Transparency; content targeting operating in an open, non-manipulative way will benefit everyone.
• The biggest struggle is in the letting go, listening and allowing the consumer to have more control.

User-Generated Content (UGC)
Presented by: Chance Parker, J.D. Power and Associates

• The UGC area of the Internet is full of publishers; people interacting with each other publishing information back and forth to each other.
• 180 million people currently maintain a blog, while nearly everyone has visited a blog, sometimes without even knowing.
• 80% are blogging to and with personal friends.
• Blogs give us access to a diverse audience: nearly 60% over 35, 20% English speaking Hispanic.
• 900,000 blog posts daily.
• Manipulation is not welcome in the blogosphere; “FLOG” = fake blog!
• Example: Wal-Mart gave content to employees to blog as consumers, to post to create buzz, and they were outed.
• Be careful of coming across like a big company; big companies tend to act like big companies while interacting with consumers. It’s difficult to engage in a personal dialogue when it sounds like the information is coming from a big company.
• Listen and learn first, before you start; take note of what’s working, what’s not?
• Surprisingly enough overall the conversations about the auto industry online are positive sentiments.
• The blogosphere is directly reflective and predictive of buying and usage trends.
• 2007 average discussion of new vehicle is higher than 2008 average buzz. “Had we looked closer at early 2008 buzz trend around automotive, it would have been very telling early on that it was going to be a trying year in the automotive industry.
• There is much value in listening to what people are saying. Just listen! Then act.

Many industry observers were stunned by the sudden collapse of Bill Heard Chevrolet, one of the country’s largest dealer groups with over $2B in annual revenue. While decreased demand and lack of credit were the most immediate causes, the dealership’s poor reputation certainly played a role in its demise. (Cliff Banks of Ward’s Dealer Business wrote an excellent article in this topic in late September.)

This turn of events brought me back to the topic of dealer ratings and the larger issue of reputation management. Bill Heard’s history of problems with regulators is well-documented – to what extent does its poor reputation carry over online? While the recent online chatter concerning Bill Heard Chevrolet drowns out previous online conversations, dealer rating sites are so far largely unaffected. The sites clearly reflect Bill Heard’s poor reputation.

On DealerRater, I found 33 reviews across multiple Bill Heard locations, with an average of 1.4 out of 5 stars. Both sales and service did poorly. Even more damaging are the reviews:

• “If I could give this dealer a zero, I would.
• “NEVER buy from or service at this dealer, royal rip-off artists!!!”
• “The price advertised was not honored. They attempted to get near this price while introducing extra charges to compensate for any price reduction to do this. They used three different sales persons over two hours to get me to agree to purchase under these conditions. Avoid this dealership entirely!”

The story on Yahoo Local was similar, with 32 reviews averaging 2.4 stars out of 5. Some quotes from the reviews:

• “I will never go back to Bill Heard.
• “I HATE THIS PLACE.
• “ . . . dealing with Bill Heard Chevrolet was a nightmare from the start. . . . salesmen badmouthing each other, a personal rivalry between the young and arrogant salesman I dealt with and the equally arrogant supervisor of the parts department, waiting three months to get the items installed on my truck that I had paid for.
• “worst buying experience ever: Lies…waited forever..inflated pricing….tried giving me my car with no gas, bald tires and w/ the AC no loger working . . .

It’s not as though Bill Heard Chevrolet didn’t know about these problems. After all, they’d met with authorities multiple times and promised to clean up their act. The group’s demise is a reminder that reputation (built on treating customers right) is necessary to the success of almost any business. And monitoring online conversations, ratings, reviews, et.al. is a great way for companies to stay on top of the issue.

We’ll be addressing this very topic at our Internet Roundtable next week, in the Dealer Ratings and Reputation Management panel Wednesday morning. In the meantime, I’d love to hear your thoughts on Bill Heard Chevrolet or reputation management in general.

I keep reading that dealer ratings/reviews are becoming important, and that dealers should encourage their customers to go online and fill them out. While I agree that consumer-generated dealer ratings may someday become an integral part of the automotive sales and service process for consumers, we’re not there yet.

People can go online to find consumer ratings/reviews on almost anything: for instance, Epinions.com has 79 different models of toasters with at least one consumer review each. A critical aspect to building any ratings is the network effect: ratings become more valuable and useful as more consumers visit and contribute. Conversely, Web sites that are unable to attract sufficient traffic and consumer ratings stall out.

For this article, I visited three Web sites with automotive dealer reviews: DealerRater, Edmunds, and Yahoo. In my survey of these sites, I searched for the top 20 dealerships from the Ward’s Dealer 500 – these high volume dealerships should be good candidates to attract consumer ratings.

DealerRater, as the name indicates, is a standalone site focused on dealer ratings and reviews. In its six years of existence, it has managed to accumulate many reviews: of the top 20 sites, eleven dealers have five or more. On the downside, many of these postings are old, indicating that the site may be losing momentum. For instance, Crevier BMW (Santa Ana, California) has 19 reviews total, but only three since January 2007. There are also other problems that hinder the site’s viability.

• No consistency in dealership names. For Ray Catena, a New Jersey-area dealership group, DealerRater gave me ten different options, including Ray Catena (2 reviews), ray catena (0 reviews), Ray Catena Motor Cars (7 reviews), Ray Catena Jaguar Porsche (2 reviews) and Ray Catena MErcedes Benz of Union (0 reviews). Note: the last is not a typo.
• More ads than information (as shown below).
• Sales and service ratings/reviews are mixed together.

On Edmunds, where vehicle research is the primary attraction, dealer ratings play a secondary role. Still, Edmunds does better than DealerRater in some respects, including separate sales and service ratings, a map showing the dealer location, and less advertising clutter. On the downside, I found fewer reviews: the greatest number was eight total reviews for South Bay BMW (Torrance, CA). Many dealers had one review or none. It remains to be seen whether Edmunds’s volume of dealer ratings will achieve the robustness needed to be a useful source of information.

For Yahoo!, ratings are an additional piece of information that gives users a full understanding of any product or service, including dealers. As shown below, the Yahoo! Local entry for Landmark Chevrolet (Houston, TX) provides contact information, hours of operation, map, photographs, and ratings/reviews. Thus, anyone looking for any information about the dealer will probably see the ratings/reviews. If there’s only one or two, so be it. As it turns out, eight of the top 20 dealers had at least five reviews, while five dealers had no reviews at all. Yahoo!’s reviews functionality has been around for over three years, so it’s clear that dealer ratings haven’t taken off here, either.

In general, ratings have evolved as useful, though still supplementary offerings that support a site’s main goals. For instance, Amazon’s consumer-generated ratings and reviews are not its focal point – instead, they support the goal of selling products. Epinions is an exception and thrives because it offers numerous reviews on a broad array of products.

DealerRater, as it is positioned currently, will be continually challenged to build enough traffic to become a popular destination. Third-party sites also have their own challenges: they attract far more traffic, giving them more chances to solicit visitors to submit a review, but these visitors may be less inclined to contribute. Both Edmunds and Yahoo seem to suffer from this problem. As it stands, there’s no single reliable source of consumer-generated dealer ratings/reviews.

It’s still too early to write off online dealer ratings/reviews, but the existing players have not yet made a compelling case for its usefulness to-date, either. There’s still the opportunity for a new entrant to get involved and really own this area, but they will need to bring something new to the game or run the risk of achieving only mediocrity.

Consumer reviews and ratings on products and services as diverse as books, hotels, electronics, and automobiles have long been an important complement (and filter) to the vast troves of online information. The network effect is key in creating value for these reviews—as more consumers visit a site and contribute their ratings, the site becomes even more useful, so even more consumers visit and contribute. This upward spiral has helped make several major third-party automotive sites become even more valuable sources of information for prospective vehicle buyers.

But online ratings are not without their biases. In order to test the differences between rating sources, OAR conducted an analysis between June 23 and June 30, 2007 based on models in six highly competitive segments. Only those models with ratings across all rating sources used in the analysis were included so that the results are directly comparable to each other. Altogether, 75 vehicles models were analyzed.

The sources for ratings were:

• ConsumerGuide: Expert ratings in 10 categories on a 100-point scale.
• Consumer Reports (obtained via MSN Autos): Overall score on a 100-point scale based on physical testing of the vehicle, drive tests, and qualitative judgment.
• J.D. Power and Associates: Paper-based surveys of vehicle buyers regarding their experiences in the first 90 days of ownership. The published online ratings are divided into three categories, each on a 5-point scale.
• Edmunds.com: Consumer-generated ratings on a 10-point scale.
• MSN Autos: Consumer-generated ratings on a 10-point scale.

Biases are to be expected with consumer feedback. For instance, market research respondents tend to avoid giving very low scores, so the average for U.S.-based survey ratings on a 10-point will almost always be higher than 5, the supposed “average” rating. Self- selecting polls and reviews (like consumer ratings) are also not representative of the target population, so the data is less likely to accurately portray the “average” owner experience. Online vehicle reviews are even less likely to be representative, since not all vehicle owners are online.

The impact of these biases can be worse when there are a small number of reviews. Consumers will often look at the number of contributors when assessing the validity of ratings. But even with as many as 20 reviews, a few strongly negative scores can have a major effect on the overall rating of a model. For simplicity, no minimum threshold was applied to this analysis.

One clear pattern is that consumer-based sources give higher scores in general without favoring any one type of vehicle. The other sources, on the other hand, give higher ratings on average to premium and import models (vs. non-premium and domestic models) than do the consumer-based sources.

These discrepancies have several possible explanations. Premium models are often packed with more features, while import models have long been held to a higher standard of design and quality, both of which would lead them to be “better” vehicles in the absolute sense. But not everyone wants or can afford these models, so perhaps consumer ratings are more reflective of “normal” people.

Additionally, the people who submit user reviews are more likely to be either enthusiastic about or extremely dissatisfied with their purchase. There’s not a lot of in-between color that would produce a bell-shaped spread of scores that reflect the actual experiences of vehicles owners. (Note that survey-based research, like that conducted by J.D. Power and Associates, generally represents a broader cross-section of experiences. The opinions of a random group of vehicle buyers is actively solicited, such that the full range of voices from positive to negative is heard.)

If enough negative voices join in, the impact to a model’s overall rating can be profound. For instance, the Volvo S60 was a top performer among MSN Autos users, but one of the worst among Edmunds users. It turns out that two Edmunds reviewers gave the vehicle scores of 1.3, pulling down a rating that would have otherwise been a respectable 8.9.

In fact, there was substantial variation at the model level among the rating sources. The chart below shows the top- and bottom-rated models for each source.

The Toyota Camry caused a notable divergence among scores: it was the lowest-rated model among consumers on MSN Autos and below average among those on Edmunds, but a top-quartile performer for expert reviews on Consumer Reports and ConsumerGuide. Many owners experienced serious problems with the automatic transmission and found Toyota’s response to be inadequate. These complaints reflect the discrepancy between expert reviewers and the real ownership experience.

Other notable divergences include:

• Lincoln MKX: Top performer on J.D. Power and Associates and MSN Autos, but below average on Consumer Reports because of gas mileage, braking, and rear visibility.
• Lexus RX Series: Strong performance on all sites except MSN Autos. Consumers gave it lower marks for performance and willingness to recommend to others.
• Infiniti FX-Series: Strong performance on all sites except Edmunds. Of 18 consumer reviews of the 2007 FX35, two were strongly negative, pulling down the overall Edmunds consumer rating despite glowing reviews from most contributors.

On most third-party automotive research sites, consumer ratings are placed immediately after price and alongside expert ratings, so the impact of just a few disgruntled owners can be profound. Any shopper who explores further will discover that the majority of owners had positive experiences, but those negative reviews may cause prospective buyers to think twice about adding or keeping a model on their consideration sets.

Consumer feedback is already a critical component of the business models of sites such as Amazon, Netflix, and CNET, while more companies are looking to leverage consumer content to supplement their own offerings. For instance, CarMax added consumer ratings in early 2007 as a component of their listings. As with all consumer- generated media, critical mass is necessary to create a virtuous cycle of contributions and visitation. Regardless of any individual site’s success, it is clear that online consumers provide a valuable complement to the other automotive research information available to shoppers.

Note: This is a condensed version of an article that originally appeared in the September 2007 version of OAR-I.

We heard snippets this morning, at the IRT, from several recently released automotive Internet research studies. J.D. Power and Associates spoke to their recently released Autoshopper.com studies and a collaboration between The Cobalt Group, Yahoo and Polk discussed findings from their Dealer eBusiness Performance Study.

While these studies shared many consistencies, one was particularly interesting – the role and importance of consumer-generated reviews to automotive shoppers.

According to the Cobalt/Yahoo study, 73% of shoppers indicated they would read user reviews and 58% said they would actually participate in these sites and write reviews. These reviews significantly influenced shoppers: 21% changed their dealership selection based on reviews, 23% considered a different brand, 43% used reviews to determine which dealer to buy from and 45% utilized reviews to confirm their chosen dealership.

Kevin Root of Cobalt discussed their use of a conjoint study to determine the importance of each aspect of the buying experience. While Reviews were solidly in the middle – behind Price and Professional Sales Staff and above Location and Loyalty – they often act as tie breakers when determining where to buy.

The J.D Power studies echoed the Cobalt/Yahoo study. They indicated that 64% of new-vehicle shoppers and 66% of used-vehicle shoppers rated “consumer ratings” as the most important aspect of consumer generated content online. When asked how helpful these reviews were, 55% said very helpful.

Consumer-generated reviews are proving to be important and influential to automotive shoppers. This is an area that all parties need to be aware of and active in, in the upcoming year and beyond.

You can download the presentations:

Cobalt/Yahoo/R.L. Polk Study target="_blank">

J.D. Power and Associates Study Recaps