Is medicare rating unreliable?  We’ve seen some commentary dismissing Medicare’s 5-Star rating system for nursing homes because it relies upon provider (community) self-reported data. Some go so far as to call the system unreliable, misleading or useless. 

See New York Times article: “Medicare 5 Star Rating Allow Nursing Homes to Game the System

Should you ignore the Medicare 5 star rating? Is the rating system fatally flawed?

Is Medicare Rating Unreliable? 

No. We think the Medicare 5 star rating system is a valuable starting point. While staffing and quality measure data is self-reported by communities, the data is cross-checked so it’s not easy to fabricate to produce misleading results. It doesn’t capture everything. It only measures what it measures. Certain state violations and complaints are not captured in the federal system. Like any summary rating you have to know what’s evaluated and what’s not evaluated. For more on how to read the Medicare 5-Star rating system, see our explanation at this link.

Medicare Rating Unreliable

Medicare 5 star rating nursing homes. Is Medicare Rating Unreliable?

For instance, staffing levels are important in the rating system but the numbers reported are cross-referenced to payroll reported to the IRS for employee compensation and tax withholding. A community can’t claim nursing personnel for Medicare’s staffing requirements without paying a nurse. It’s like double-entry bookkeeping. A community cannot easily lie about nursing staffing levels.

Similarly, on Quality Measures (QMs) a facility can’t get paid by Medicare or Medicaid without itemized services that are related to the QMs. Care for a bedsore would show up in the reimbursements and the bedsore would show up as a negative outcome on Quality Measures. Again, that idea of double-entry bookkeeping.

To further assure the accuracy and reliability of self-reported provider data, Medicare is instituting spot inspections of both state regulators and nursing facilities. If the spot inspections reveal big gaps between self-reported data and actual observed results, expect further reform.

The rigor of Medicare’s evaluation process is increasing with each successive version or iteration. The increased use of spot inspections is new in 2015.

And remember, even with self-reported data, there’s still a wide spread in performance. The rating system forces a distribution within a state as within each category only a limited number of providers can be rated 5-stars (it varies by category, but generally only the top 15-20% earn the 5-Star rating).

The use of self-reported data is not a fatal flaw. Nor is the exclusion of other data or factors to consider.

The biggest flaw in our mind is that the rating is on a state-by-state basis and doesn’t make comparison across state lines easy.

Medicare 5 star rating system is a good starting point. The use of self-reported data is not a fatal flaw. Nor is the exclusion of other data or factors to consider.

For instance, while nursing staff levels are evaluated by Medicare, other staff jobs may play a big role in how a resident experiences everyday life in the community. Medicare doesn’t rate housekeeping and custodial or food service workers. It doesn’t mean they’re not important. It means you still have to ask current residents and family members of residents how they feel about the quality of care and service. We repeat this admonition often, never rely solely on staff to tell you about a facility. Even well-intentioned staff are going to be biased. And no prior evaluation is perfect. Even a great facility will occasionally have a bad outcome. And frankly, facilities are dynamic. They change over time. Some get better. Others decline. You have to continue to pay attention.

There’s no substitute for intense research. Don’t stop at the Medicare rating. But it’s a useful filter to limit the number of choices you’re considering. It’s not a guarantee of satisfaction.

Read Two Closely Related Articles:  How to read the medicare star ratings and Medicare Star Ratings