Metrics and Evaluation Plan
Plans for Evaluation
Georgia will conduct a comprehensive program evaluation across the five initiatives. The focus of the evaluation plan will include overall outcomes per initiative as well as opportunities for improvement throughout program implementation. Georgia will cooperate with any CMS-led evaluation and monitoring and recognizes that CMS and/or third-party evaluators may assess outcomes across the States. The state will contract with Georgia State University (GSU) to study the program's impact on the health of the state. GSU’s strong history of federal and state-specific evaluations and an understanding of program implementation. The state and GSU will work closely to produce a comprehensive evaluation plan during Year 0.
The GREAT Health Program will use a mixed methods Implementation Science Framework for evaluation, called RE-AIM. RE-AIM has historically been used in large-scale innovative implementations to measure the effectiveness of funding mechanisms to reach desired outcomes, as well as information about how positive outcomes were achieved, and how outcomes can be replicated in other similar circumstances. RE-AIM assists evaluators in considering and formulating plans to measure reach, effectiveness, adoption, implementation, and maintenance among large scale implementations. Measuring outcomes through RE-AIM will allow evaluators to produce reports that can lead to year-to-year modifications for programs who may be underperforming and understanding needs of the various initiative strategies.
Using the RE-AIM framework, the evaluation team will conduct annual reporting and create data dashboards per initiative to monitor progress at least bi-annually. Evaluators will ensure in Year 0 that baseline measures and annual goals are submitted from sites to ensure program alignment with predicted targets. The evaluation team will confirm appropriate data mechanisms are used to develop baseline and goals, and all data sources will be documented to ensure consistency in sources and collection methodology across all aspects of the program.
Data sources include, but are not limited to, the All-Payer Claims Database (APCD), a statewide repository that collects claim data from public and private employers, as well a Electronic Medical Record (EMR) Threat Detection and Response systems, federal level data tracked through HRSA, and other available sources throughout the state, such as annual reports from sub-grantees and hospital surveys. Program partners and grantees will be required to submit outcome metrics twice a year to gauge improvement and barriers each year. Plans for data aggregation will be finalized as part of the comprehensive evaluation plan during Year 0. In addition to quantitative data collection, the evaluator may also conduct qualitative data collection strategies to reflect stories of impact, such as focus groups with hospital leaders, providers, and rural community members. The evaluation team will meet at least quarterly with the Program leadership team to understand progress and needs and create plans for technical assistance.
Overall GREAT Health Program Project Outcomes:
The overarching GREAT Health Program outcome metrics below align with the program vision of healthier populations, abundant access to care in rural places, and leveraging technology to drive progress in rural areas by FY31.
- Increased access to 10% of rural residents measured by increased use of telehealth services and reduced travel time and distance to services
- Decrease in all-cause mortality in rural areas by 15%
- Decrease in readmissions in 75% of rural hospitals
- 20% increase in the ratio of rural primary care providers to rural population
- 0% increase in the number of patients in participating rural counties with preventative screenings at evidence-based intervals
Initiative Key Outcome Metrics and Addressing Overlapping Metrics
Each initiative has key performance outcome metrics that will indicate the impact of the project on the health of Georgia. The initial evaluation plan, which includes metrics for each of the five initiatives, is in Attachment I: Evaluation Plan Initiative Metrics. In year 0, a comprehensive evaluation plan will be developed that will also include metrics for each strategy associated with the five initiatives to ensure the strategies are on time and target.
Georgia acknowledges that all health is interconnected; as such, metrics overlap between initiatives. This overlap represents the goals of the larger project at utilizing multiple streams of investment to provide a measurable and sustainable outcome. The first metric in multiple initiatives is year-over-year improved physical health outcomes for populations receiving care in participating hospitals. This metric is found both in Transforming for a Sustainable Health
System and Strengthening the Continuum of Care. Improved physical health outcomes for large scale populations will require several different strategies to make measurable differences in rural areas in the state, as poor health is not determined by one factor. The Transforming for a Sustainable Health System initiative focuses on preparing to implement the AHEAD model in hospitals throughout the state. Simplified reimbursements models dependent on value-based care are imperative to population health and should have an impact on health outcomes in areas where initiatives are taking place. Similarly, Strengthening the Continuum of Care has several strategies that focus on improved outcomes for different populations, including but not limited to children with behavioral health challenges through the Building Bridges strategy, newborns through the Newborn Screening strategy, and children with ASD through the nutrition support services strategy.
The second metric in multiple initiatives is decreased emergency department visits for non-emergency concerns in rural counties. This outcome falls under both Connecting to Care to Improve Healthcare Access and Leveraging Technology for Healthcare since strategies target rural access to care for preventative health services, alleviating unnecessary use of emergency services. The Connecting to Care to Improve Healthcare Access initiative has multiple strategies strengthening capabilities of rural health entities to bring increased preventative care access to rural populations with the intention of decreasing the use of emergency care services. These strategies focus on mobile units, building and strengthening rural telehealth service infrastructures, and uniting health entities to coordinate systems of care to get the right patient to the right location at the right time. Complementing these strategies, those in the Leveraging Technology for Healthcare Innovations initiative strengthen the EMR systems of hospitals and invest in consumer-facing, care-enabling technologies. EMR system improvements, as preparation for the AHEAD model, give insights into identifying at-risk patient populations.
Technology company investments will address the burdens of chronic disease and care access in rural areas. Both strategies display, alert, and educate providers (EMR system enhancements) and patients (consumer engagement enhancements, ARCHER fund innovations) alike on trends and habits, flagging concerning patterns before requiring ED support.
The third metric increases use of primary care/preventative care services related to physical health, chronic disease, and mental/behavioral health. This outcome falls under both Growing a Highly Skilled Healthcare Workforce and Leveraging Technology for Healthcare Innovations, since strategies target rural access to care for primary care and preventative health services. The Growing a Highly Skilled Healthcare Workforce initiative will increase and strengthen the rural health workforce by incentivizing health professionals to practice in rural areas, improving access for those living in surrounding rural areas. While incentives vary, many include strengthening the technological innovation in rural areas, reducing the burden and cost of opening a practice in rural areas, allowing practices to have a wider service area through telehealth services, and providing similar technologies to those in urban training centers, such as robotic surgical instruments. The intention of both initiatives is to improve access to primary and preventative care and see an increase in visits related to improving management of disease, particularly chronic disease and mental/behavioral health.