Methodology

The scoring and ranking methodology in the 2023 LTSS State Scorecard is substantially revised from the methodology used in the previous four Scorecards.  The most significant change is that in earlier editions, states were scored based on their rank on each indicator; in the current Scorecard, they are now being scored based on the value, or actual level of performance, for each indicator.

Dimensions and Indicators: The 2023 Scorecard measures LTSS system performance using 50 indicators across five dimensions.  Indicators consist of metrics, which have a numerical scale from best to worst performance, and policies, for which states are credited for having such a policy, or not (some policies have a single intermediate “partial credit” option where it makes sense).  The 50 indicators (30 metrics, 20 policies) are grouped into the following dimensions:

• Affordability and Access (6 metrics, 1 policy; 7 total indicators) 
• Choice of Setting and Provider (8 metrics, 3 policies; 11 total indicators)
• Safety and Quality (9 metrics, 4 policies; 13 total indicators)
• Support for Family Caregivers (1 metric, 11 policies; 12 total indicators)
• Community Integration (6 metrics, 1 policy; 7 total indicators)

Indicators had to be important, meaningful, understandable, have a clear directionality, and have comparable data available at the state level. These 50 indicators were selected because they represent the best available measures at the state level. While no single indicator can fully capture LTSS system performance, taken together they provide a useful measure of how state LTSS systems compare across a range of important dimensions. 

Scoring Metrics.  Raw metric values are transformed to a natural scale, including reverse coding for metrics where a lower value is better.  In general, percentages undergo a log odds transformation, supply and other ratio measures undergo a log transformation, and other measures are only adjusted for directionality.  Table D.1 shows the transformation used for each metric.  The transformed values are then standardized to a “Z score” with mean of 0 and a standard deviation of 1 across all states, so that for scoring purposes each metric has the same weighting as every other metric in the dimension.

Equity Indicators.  Nine metrics are equity indicators, where instead of being scored on the metric value for the entire population, states are scored only for the value of the worst performing racial/ethnic group. 

• Five of these metrics divide the population into two groups (non-Hispanic White; and a combination of all other race/ethnicity groups: American Indian/Alaska Native, Asian, Black, Hispanic, Native Hawaiian/Pacific Islander, and multiracial).  Where sample size is sufficient for both groups, the indicator value is the lowest of the two groups. 
• The other four metrics are calculated at different unit of analysis (nursing facility or neighborhood); for these, the value for the 10% with the highest proportion of residents of each identifiable race/ethnicity group is calculated.  The indicator value is for the worst performing 10% subsample.

Scoring Policies.  Policies are scored 1 for “full credit” for the indicated policy, and 0 for “no credit.” This is not necessarily the same as having a no policy.  For policy indicators in which an intermediate level of credit makes sense, “partial credit” of 0.5 may be assigned.

Innovation Points.  Six policy indicators are identified as “innovation points” to call attention to policies that only a few states have adopted and thus have potential for significant LTSS system improvement if implemented more widely across the country. The scoring of innovation points is the same as for all other policy indicators.

Calculating Dimension and Overall Performance.   Dimension-level performance is calculated by summing the metric Z scores and policy scores for all indicators in the dimension.  In order for dimension performance to be based on total performance across many indicators and not dominated by outlier high or low performance in a single metric, metric Z scores are capped at 2 and floored at -2 when summed.  A higher dimension score is considered better performance.

Every metric has equal weight in determining dimension performance, and every policy has equal weight.  Full policy credit (relative to no credit) is equivalent to one standard deviation difference in metric-level performance.

Overall performance is calculated by summing standardized dimension-level performance, so that every dimension has equal weight in determined overall performance.  Dimension Z scores are capped at 2 and floored at -2 when summed. 

Performance Tiers.  At the dimension and overall levels, states are categorized in performance tiers from Tier 1 (best performance) to Tier 5 (worst performance).  These tiers provide more context about state performance that individual state ranks. Tiers 2, 3, and 4 represent equal performance ranges, with Tier 1 and Tier 5 showing exceptionally high or low performance.  Cut points of approximately +1.5, +0.5, -0.5, and -1.5 standard deviations from the mean are used to classify states by tier (some cut points are adjusted slightly from default values so that there is always a meaningful performance gap between each tier and similar performing states are not separated into different tiers).

Table D.1