The CR survey methodology, at best, should be taken as informational. It does not follow rigorus statistical methodology.
The sample size is small and is sourced from a captured group. Result data is poorly structured and totaled up in a catch all way and called reliability.
I believe readership has been nearly halved in recent years, rendering an even more captured group.

Perhaps you regularly read the CR surveys. Here's a few points...
I live in sunny CA; rust is not an issue to me. Not so true of my friends in the great Mid West. Are survey results geographical? Do we know where the numbers come from?
Makes can make great ranking changes from year to year. How does, BMW for example, go up 7 (or 10) spots in 1 year? Are all the cars magically that much different?
Why are their rankings out of synch with their owner satisfaction results? Do people love garbage "relaibility" and hate great "reliability"? That's what the CR survey results are telling us, in many cases.

My career was in business forecast for billion dollar companies; statistics are key. But you have to look at the raw data and read into it. The #1 parameter is sample size; small numbers and source must be shown to be relevant or you will draw incorrect conclusions.

 

You bring up some good points about different experience in different areas. Still, CR’s reliability-database is one of the largest in the auto business.

 

The mentioned biases in CR data is completely legitimate. To me its a question of what does the general picture from multiple sources look like.

Take Chrysler, every measure rates them at or near the bottom, CR, JD Power, pretty much everything and you see that lived out in forums and in my own experience owning the brand. I have total confidence when I tell people Chrysler doesnt make consistently high quality vehicles, and that they should be prepared for negative experience. Same is true of Toyota and Lexus on the other spectrum, they are rated at or near the top of every measure, and forums and my own experience lives that out, and I have total confidence when I tell people Toyota/Lexus makes consistently high quality vehicles and they they likely will have a great experience.

But, in the middle it gets murky. My experience with Mercedes has been excellent, but CR right now rates them very low, JD power rates them much better. Specific models CR rates as well below average JD power rates well above average (such as the GLE and E Classes), and my experience on the forum shows some models seem to align with CR (the GLE) and some with JD power (the E Class)...

 

You are right. There is bias in any survey or research. But Most data supports CR findings for the most part. Word of mouth too. You haven’t owned a Mercedes out of warranty or higher miles,

 

No but I know people who have, and I post and read a lot on the MB forums like I do here across all the models and the result is, some models are really good, and some are not. It’s not as simple as saying that Mercedes as a brand makes reliable cars or doesn’t.

 

This is a perfect example of the CR survey fallacy. CR tested 2 MBZ models. How do you honestly rate a car manufacturer on 2 models? In a statistical report, this would be laughed at. And to make a big $$ decision on this these MBZ results woud be foolish.

 

Thats a completely legitimate criticism.

 

Can you support your post with some data? CR's own findings rate "reliability" tend to disagree with their own customer satisfaction numbers.
I also disagree with bias. A valid statistical model's goal is to elimanate bias. And beyond the raw numbers, you need to examine the results to validate them. I see none of this in the CR survey which is why I consider it informational at best.
As an example, surveys often look at age as an important consideration. It would be difficult to gain MBZ valid data from 18 year olds.

 

Right. So I just go by with family and friends word of mouth, then I see if it backs the publications and their opinions. . My friends S class like yours (but not same MY) has had some issues. Hesitation with the engine and start stop. And problems connecting phone. She still loves it. But it has spent a lot of days off the road for diagnosis. And my friends newer Golf top spec engine model burns coolant or is always low. But he loves it too. And it’s leagues ahead of what Toyota or GM has at that price point.

 

You have to look at all of the different sources to get an idea of what the situation is, can’t rely on just one source…basically exactly what you said lol

 

Exactly. And then just buy what wifey wants. At least that's what I do....

 

[QUOTE=JeffKeryk;11400279]This is a perfect example of the CR survey fallacy. CR tested 2 MBZ models. How do you honestly rate a car manufacturer on 2 models? In a statistical report, this would be laughed at. And to make a big $$ decision on this these MBZ results woud be foolish.[/QUOTE


If I understand it correctly CR would not rate the whole MBZ line on the results of two models (road) tested. Those test results would just rate/rank those two specific models in their respective segments. Their reliability ratings and brand rankings, right or wrong, would be drawn from their subscriber surveys which would include various models.

 

To add to that, consistency of results year after year. CR has been especially bad at this.

I said it a year ago, but CR's sample sizes following COVID-19 has become much more sparse than before. That explains why there is more variation in data, and brands will jump up and down 10 or 14 spots without any major change in their lineup. BMW went up 10 spots this year but may very well go down by 14 the following year.

JD Power's Dependability survey has been much more consistent year-to-year than CR.

 

[QUOTE=LexBob2;11400335]The MBZ ranking was based on 2 models tested in the current CR survey. That's bogus.

 

[QUOTE=JeffKeryk;11400392]I agree that the way they do things is questionable, but the road tests which CR does and owner reliability surveys which are mailed to owners are two separate things. The performance ratings from the road tests and survey reliability results are also two separate things. I'm not a subscriber but was interested in how they come up with their stuff.