Consumer Reports Hate Factor
After I read that Consumer Reports was not recommending the 2011 Odyssey I decided to dig into the CR numbers some more to see if there is something other than pure objectivity in the ratings. I did the analysis as follows:
1. The CR ratings are based on a 5 point scale where the solid red circle indicates much better than average and the black solid circle indicates much worse than average. In order to compare the ratings I converted the color codes to a number so I used 1,2,3,4 and 5 to represent worst to best. I then went through the CR 10 year reliability ratings (I have an online subscription) and manually entered a score for each rating of each of the reliability characteristics they use (e.g., engine major, transmission major, suspension, body hardware, paint etc.). There are a total of 17 characteristics and for each historical year there is a "verdict rating" based on thousands of consumer inputs. There is also an extra row in the data where CR provides an overall rating (between black and red or 1 and 5 respectively). I started with the assumption that CR has been getting progressively "emotionally" biased towards the Odyssey over the last few years. To test that hypothesis I took the sum of all 17 of the ratings for one particular year and calculated an average. Then for that same year I converted the CR colour code to a number and compared the two. I did this for all of the available years (2002 through 2011) and summed the differences to arrive at what I have coined an annual CR Hate Factor (AHF). If the value is negative then it could arguably be called the Love Factor.
2. Based on my calculations the annual CR Hate Factor was zero (no hate, no love) from 2002 through 2005. After that the Hate Factor became consistently positive, with the value being +1 from 2006 through 2010 and then +2 in 2011. I don't know the exact scoring method that CR uses prior to converting to the 5 point color codes but I can imagine there may be some mathematical rounding going on here. That being said, even if I remove every other year from the calculation there are still 3 years where the numbers don't jive and for some reason CR rated the Odyssey worse than the numbers suggest. If you assume that hate is cumulative then summing each annual hate factor gives an Overall Hate Factor (OHF) of +7.
3. More troubling than this is the fact that the calculated rating for the 2011 Odyssey using the CR color codes is above average which corresponds to a numeric value of 4. However CR chose to rate the 2011 as below average or 2. They must be really hating the 2011 (AHF = +2...see above). Who at CR has shares in Toyota?
4. I suppose that maybe CR is using some kind of weighting to rate some of the 17 reliability characteristics as more important but this defies logic. In my opinion more important characterstics are things like "Engine Major", "Engine Minor", "Transmission Major" and "Transmission Minor". Looking at the 10 year data, all of these characteristics are BETTER between 2006 and 2011 not worse. What gives with that? There have been some problems with body hardware and electrical on the 2011s but those in my mind are not as serious as a transmission failure or an engine failure.
5. I've done some weighting analysis on my own and when I weight the Engine/Transmission/Fuel Supply higher than the other characteristics the reliability ratings get even better (except between 2002 and 2004 with the tranny) and the OHF increases to between +11 and +12. At this stage I'm not going to send any of this to the newspaper or notify Honda that they are getting bad/inaccurate reviews from CR. I am going to do this analysis with some other vehicles that CR has reviewed and see if this is an anomaly with the Odyssey. If anyone is interested in looking at my calculations let me know and I'll send you a copy.
Watchman