ABBREVIATIONS

CSRS, coalition sustainability readiness strategy; EBI, evidence-based intervention

BACKGROUND

Community coalitions are increasingly used to address varied health issues [1–3]. This community infrastructure consisting of multiple organizations brings together diverse groups/organizations to affect community change on complex health issues [4]. Major initiatives funded by the U.S. federal government and large foundations often require a coalition structure [5]. Coalitions address health challenges including: immunization [6], substance use prevention and treatment [7–9], adolescent multiple risk behavior [10], obesity prevention [11,12], physical activity [13], emergency preparedness [14], and health disparities [15–19]. Health promotion efforts have produced positive outcomes in health sectors that include immunization [6], substance use prevention [20,21], mental health [22,23], and the capacity of community practitioners to implement positive youth development-oriented prevention practices [24].

Coalitions are key mechanisms for implementing health-focused evidence-based interventions (EBIs) [5,25,26]. Coalitions are a promising strategy for supporting EBI implementation, partly through attaining support from key stakeholders and ensuring that EBIs are implemented with sufficient dosage and fidelity [27]. For example, the Communities That Care system involves coalitions that implement EBIs to address adolescent substance use and delinquent behavior. This system has demonstrated sustained impacts on behavior and on sustaining prevention programs [20,21,28].

While the literature and funders often call for organizations to sustain EBIs including coalitions, we conducted a scoping review from 2010 to 2023 [29] that found only three articles presenting a health-focused sustainability strategy [30–32]. Our scoping review included a search of nine databases within EBSCOhost plus other searches in Google Scholar and PubMed. An expanded search to early 2024 yielded two added articles [29,33], the former was conducted by us and is the only publication that focused specifically on EBI sustainment, and it presented interactive tools and an evaluation component to assess readiness and intentions to sustain EBIs. Our sustainability strategy for evidence-based health interventions is described in detail in our earlier publication [29] and the PIRE Louisville Center website [34]. Here we describe a revised strategy for coalition sustainment that addresses the limited strategies available to continue coalitions beyond initial funding. Later we present research questions for a future implementation study.

INTERVENTION

The proposed coalition sustainability readiness strategy (CSRS) for communities is based on scientific evidence. This includes: (1) a research-based conceptual framework, (2) a data-driven, decision-making model with a step-by-step toolkit and interactive tools including an automated evaluation system, and (3) virtual coaching, which research has found to be comparable in effectiveness to onsite coaching [35–38] and more cost-effective [36].

Our conceptual framework is informed by the literature on the importance of readiness and the R = MC² readiness framework [39]. Readiness refers to the capacities and willingness of an organization to engage in a change effort; it has been accepted as a necessary precursor to successful implementation of innovations across the implementation science literature. The R = MC² framework synthesizes what is known about readiness, explicitly calling attention to motivational and capacity aspects of readiness for change; it defines readiness (R) as the organization’s motivation (M), general capacity (C), and innovation-specific capacity (C) (abbreviated as R = MC²) to implement the innovation. Motivation refers to the willingness and momentum of implementation; general capacities refer to the structure and functioning of an organization overall; and innovation-specific capacities refer to the capacities needed to implement a specific innovation (i.e., program, policy, practice, or process).

Table 1 lists and defines five motivation factors (compatibility, EBI champions, simplicity, sustainability champions, and trialability) and five infrastructure capacity factors (data resources, expertise, formalization, funding resources, and policies) that predict the sustainability of health EBIs including coalitions. Since our focus in this article is only on sustaining coalitions as organizations, we did not include innovation-specific capacity factors.

TABLE 1

Table 1. Readiness factors that predict sustainment of coalitions.

Table 2 presents 14 studies of which nine (64%) were published in the last 10 years. The top part of Table 2 presents the readiness factor(s) associated with sustainability outcome(s) in each study. Eight studies show significant relationships between one readiness factor and a sustainability outcome. Six studies show a significant relationship between more than one readiness factor and sustainability outcomes. The lower portion of the table shows the health intervention, sample size, and the analyses used for each study. These analyses show an association between readiness factors and sustainability outcomes.

TABLE 2

Table 2. Studies showing relationships between sustainability readiness factors & sustainability outcomes.

The conceptual framework shown in Figure 1 illustrates the interrelationships of evidence-based causal factors and the CSRS outcomes. Reading from left to right, motivation for coalition sustainability and organization infrastructure capacity are barriers to achieving long-term sustainment of coalitions. Four key resources supporting CSRS implementation are shown around the box containing the strategy. These include a Toolkit, Excel™ Tools, a Coaching Guide, and an Evaluation Guide. This readiness strategy should impact the short-term outcomes. If achieved, this should impact the intentions and actual sustainment of coalitions.

FIGURE 1

Figure 1. Coalition sustainability readiness strategy conceptual framework. Note: This figure has been modified from “Developing a sustainability readiness strategy for health systems: Toolkit, interactive tools, and virtual support system” by Johnson K, Collins D, Wandersman A, 2023, Evaluation and Program Planning, 97, 102241 (https://doi.org/10.1016/j.evalprogplan.2023.102241) [29]. Copyright 2023 by Elsevier.

The coalition sustainability strategy is data-informed. Prior studies show that data-informed decision-making can assist practitioners in identifying weaknesses and planning improvements [53,54]. In our literature review, we found three toolkits (listed below) having a step-by-step process aligned with an overall conceptual framework: the Guide to SAMHSA’s Strategic Prevention Framework [55], the Fuld Institute Evidence-based Implementation and Sustainability Toolkit for Health Care Settings [33], and the Getting To Outcomes^® (GTO) Manuals (published by RAND) [56]. Of these, only the GTO Manuals (e.g., Getting To Outcomes^® Guide for Teen Pregnancy Prevention) and the Fuld Institute toolkit included tools to be implemented as part of the steps. Many toolkits were only compilations of resources.

A continued literature search of health-related sustainment found only two articles that presented interactive tools. One is an adaptation of Getting To Outcomes^® (GTO) with interactive step-by-step tools [29]. GTO is an evidence-based process model that has been used successfully to address the implementation of health interventions [57–59]. This model builds capacity for implementing evidence-based interventions by strengthening knowledge, attitudes, and skills needed to choose, plan, implement, evaluate, and sustain interventions [57]. The second is the Program Sustainability Assessment Tool (PSAT), which provides data from an online, interactive tool as the first step of a sustainability process introduced in later, in-person training and technical assistance (TA) [31,60–62].

Our CSRS steps incorporate 10 implementation questions in Table 3 that are adapted from the original GTO questions focusing on implementation of health interventions.

TABLE 3

Table 3. Ten CSRS implementation questions.

The CSRS process is introduced to users in a revised toolkit as the key support resource. The coalition is asked to appoint a two- to three-member leadership committee and a workgroup (ideally three to five members). The leadership committee should include several key coalition leaders, for example, coalition director and/or committee chairs. Workgroup members should be knowledgeable about community health issues and committed to doing what it takes to achieve sustainability. Members should have skills in gathering and analyzing information and in communicating and promoting sustainability actions of the CSRS to the entire coalition. The leadership committee and workgroup will collaborate with an external coach to complete CSRS toolkit tasks over 10 months. A coach, workgroup facilitator, and workgroup data coordinator are trained in the CSRS use of Microsoft Excel™ and electronic data capture system, which is described later. Some Excel™ information collected by the sustainability GTO questions pre-populates the fields of related questions that follow. Formulas in the Excel™ tools calculate sustainability readiness and intention adequacy change and thereby report results back to the survey recipients. The Excel™ tool provides immediate feedback and reporting back to the work group.

The workgroup facilitator and data coordinator need to have basic Excel™ expertise to manage data-processing-related tasks. The CSRS includes other features to help coalition workgroups complete the Excel™ tools throughout CSRS implementation. These include (1) training in the GTO process; (2) participation of the data coordinator in virtual meetings; and (3) provision of TA by the external coach in virtual meetings. The data coordinator manages the survey and Excel™ data tasks for the entire GTO process. The workgroup facilitator presents interim and final results to a leadership committee and coalition membership for review and comments.

The CSRS implementation takes place within six virtual meetings over a 10-month period. These meetings include a startup meeting to address the 10 sustainability GTO questions focusing on CSRS implementation. Sustainability readiness and intentions to sustain the coalition are assessed at baseline, post-intervention, and six months after implementation. Results are delivered to the leadership committee for discussion. Table 4 presents this implementation process, including meetings and evaluation tasks.

TABLE 4

Table 4. Coalition sustainability GTO implementation and evaluation.

The sustainability leadership committee and workgroup participate in Meeting One, led by a coach. Content is delivered through PowerPoint™ presentations. The coach provides consultation after this and each subsequent meeting, and the workgroup facilitator provides a summary to the leadership committee.

Sustainability GTO Question 1 is addressed by the workgroup data coordinator. S/he provides a link to an online pre-readiness survey to all coalition members to collect data and assess readiness and intentions to sustain the coalition. The survey includes items that measure the readiness outcomes in our conceptual framework. Items include scales and indexes from PIRE’s Tennessee Strategic Prevention Framework State Incentive Grant (SPF SIG) evaluation [63], sustainability study [8], and additional sustainability studies. The data coordinator enters the survey results into an Excel™ tool that calculates adequacy scores for all readiness and intentions outcomes.

Workgroup members address sustainability Question 2 in Meeting Two. This involves converting inadequate and marginally adequate baseline readiness scores calculated in Excel™ to specific outcome(s) statements.

In Meeting Three the workgroup selects readiness actions to improve each inadequate or marginally adequate readiness score to address Question 3. The workgroup also uses consensus to address each selected action’s fit (Question 4). The assessment of fit of readiness planning actions addresses the challenge of alignment with a coalition’s needs and capacities [64,65]. Finally, the workgroup identifies resources needed to address each readiness action selected (Question 5).

In Meeting Four, the workgroup prepares a written plan using an Excel™ tool to address Question 6. The plan outlines tasks to be performed and identifies the lead person and other key people needed to implement each task as well as the due date.

Meeting Five addresses Question 7. It involves monitoring implementation of all readiness action tasks in the written plan. The workgroup arrives at consensus for the level of success of implementation. For Question 8, the data coordinator provides coalition members with a link to an online survey of sustainability readiness and intentions. The survey provides interim data, and the data coordinator enters it into an Excel™ tool that calculates adequacy scores for each outcome.

Meeting Six addresses Question 9 to assess Continuous Quality Improvement (CQI). The workgroup addresses readiness and intentions outcome(s) that are not adequate as well as their confidence to deal with any inadequacy(ies). CQI is important to bringing about change, including in healthcare [66].

Six months after CSRS implementation, the data coordinator provides a link to an online survey through which coalition members respond to coalition sustainability readiness and intentions outcomes items. For Question 10, the data coordinator provides a report showing the adequacy of readiness and intentions and the leadership committee decides whether to continue moving forward to ensuring coalition sustainment.

A third evidence-based CSRS feature is webinar-based coaching. Literature shows that web-based coaching is comparable in efficacy to in-person coaching [35–38]. Web-based coaching also has the advantage of being more able to reach organizations (including coalitions) that are located in different sites [35]. Online technologies have been shown to be more cost-effective than other methods [67], and research suggests that virtual (web-based) coaching is more cost-effective than in-person coaching [36].

The coach prepares for each workgroup meeting by developing a plan that includes each action to be implemented in the meeting. For each action, the coach uses the talking points in the PowerPoint™ slides. The coach refers to added content from the Toolkit and presents and leads discussions of the content. The workgroup then completes tools using consensus. The coach will remain in each of the six meetings to answer questions.

We will include in the CSRS an automated evaluation system. The CSRS will incorporate a cost-effective evaluation that includes: (1) an automated evaluation system and (2) an Evaluation Guide for the Data Coordinator, who will coordinate data collection and use interim results from the automated system to enter results into the Excel™ tools and produce reports for the workgroup and leadership committee. REDCap (Research Electronic Data Capture), an electronic data capture system [68], will be used to handle all CSRS survey data collection and reporting functions. REDCap (1) allows coalition members to complete CSRS surveys as online survey forms, (2) calculates measures that assess readiness and intentions to sustain the coalition and provides reporting to workgroup members through online reports, and (3) serves as a user interface for the data coordinator through which he or she can access surveys and reports. While there is no research supporting our evaluation services, we believe it is essential to provide CSRS implementation results throughout the sustainability process. Our review found no prior strategy that included an automated evaluation of the sustainability readiness process with an evaluation guide for a coalition data coordinator.

FUTURE STEPS AND CONCLUSIONS

The CSRS consists of a toolkit and an evidence-based, data-informed process with virtual coaching and automated evaluation. The goal is to increase coalition membership motivation and infrastructure capacity to produce adequate coalition readiness for sustainment. This strategy should lead to coalition sustainability.

We propose three steps before scaling up and diffusing the CSRS to other community coalition networks. The first step is to revise published EBI sustainment tools [34] for a study of community coalition sustainment. The tools will be revised to focus on coalition sustainment rather than EBI sustainment.

Our second step is to pilot-test our CSRS in a sample of community coalitions that have been in operation for six to eight years using start-up funding and have a supportive staff. Research questions addressed focus on what are: (1) changes in short-term outcomes, (2) changes in long-term outcomes, and (3) implementation qualities of the CSRS. This study is needed to demonstrate the strategy’s impact to address coalition readiness. Further, it will show that the strategy can be implemented by health practitioners. A follow-up study is needed to determine impact on coalition sustainment.

Quantitative data from the CSRS’s automated system provides data for a three-wave longitudinal analysis from baseline to post-CSRS implementation to a six-month follow-up. This analysis measures change in the 10 short-term readiness outcomes and the long-term outcome of leadership intentions to sustain a coalition presented in Figure 1. A qualitative assessment is needed to assess implementation quality (e.g., reach, dosage, and fidelity) [69]. This assessment will be conducted after implementing the CSRS. If the implementation study produces positive results, step three would entail scaling up other interested community coalitions that meet the selection criteria of the implementation study.

For the third step after the implementation study, a diffusion and social marketing strategy, as promoted by Dearing and colleagues [70,71], will be used in scaling up the CSRS to community coalition networks. Dearing and others have advocated the convergence of these dissemination strategies using (a) diffusion to emphasize use of existing communication channels of persuasion and (b) social marketing that advocates creating other communication channels. These authors have presented principles to guide the convergence of diffusion and social marketing strategies [71] in spreading our CSRS. The details of our diffusion and social marketing strategy are forthcoming.

In conclusion, our proposed CSRS (1) is strategic, (2) comes with practical tools, and (3) provides virtual coaching and automated evaluation. Dissemination to other coalition networks follows successful implementation of the pilot study described.

DATA AVAILABILITY

No data were generated.

AUTHOR CONTRIBUTIONS

Knowlton Johnson and David Collins wrote the initial draft of the article; Knowlton Johnson, David Collins, Stephen Shamblen, and Abraham Wandersman contributed to revisions of the article.

CONFLICTS OF INTEREST

The authors declare that they have no conflicts of interest.

FUNDING

This work was supported by a contract with the Southeast Center for the Application of Prevention Technologies (CAPT) and the Center for Substance Abuse Prevention (CSAP); the CSAP-funded Tennessee Strategic Prevention Framework State Incentive Grant (SPF SIG); and internal funding including in-kind funding from the Pacific Institute for Research and Evaluation (PIRE) and the University of South Carolina Department of Psychology.

ACKNOWLEDGMENTS

The authors want to thank Bill Wieczorek, Roland Moore, and James Dearing for reviewing drafts of this article and providing feedback. We also appreciate the helpful comments provided by the reviewers. Finally, we acknowledge the assistance of Marilyn Thomas and Sharon Collins in preparation of the manuscript. Table 1, Table 2 and Figure 1 are modified from “Evaluation and Program Planning, 97, Johnson K, Collins D, Wandersman A, Developing a sustainability readiness strategy for health systems: Toolkit, interactive tools, and virtual support system, 102241, Copyright (2023), with permission from Elsevier.