Complete PQI Tool Kit for Print.pdf -

Please enable JavaScript to view the full PDF

Read Me First Performance and Quality Improvement Tool Kit Welcome! This is version 1.02 of COA’s PQI Tool Kit. We hope that you find this information helpful for you and your organization. In order to make appropriate updates, we need your input to tell us what works, what doesn’t work and how we can improve it. We ask that you consider taking a very brief survey about the usefulness of the Tool Kit. It should take no more that 5-10 minutes and will guide the future development and revision of the PQI Tool Kit. Along with improvements, we are also seeking new ideas for tools that can be part of this kit. This initial version includes relatively simple tools to help organizations get off to a great start to PQI. We recognize that some organizations may benefit from more advanced PQI tools. They will appear in future versions, and with some feedback from our stakeholders, we hope to make it an even better resource. Taking the survey is a great way to provide feedback on the PQI Tool Kit. However, you can also email David Haynik @ dhaynik@coanet.org. All feedback is welcome, whether it is technical or related to content. To access the survey, please click the link or go to the website by typing in: http://coa.formassembly.com/232020 Originally released on September 21, 2016 Version 1.02: Revised on November 21, 2016 Revisions include the addition of Section 14 “Additional Guidance” which supports organizations who need to provide additional evidence of implementation to COA. Council on Accreditation The Performance and Quality Improvement Tool Kit A comprehensive guide to developing a performance and quality improvement program Version 1.02 Revised on November 21, 2016 Section 1: Main Document Table of Contents Performance and Quality Improvement Tool Kit Using the Table of Contents All of the tools and example documents are accessible through the table of contents below. These resources are only available through the PQI Tool Kit table of contents and cannot be obtained through the public Tool Search. Please use the direct links below to access the full list of resources. All tools and examples are available in PDF version only. Table of Contents # Title Related Tool Example Tool 1 Introduction None None 2 Culture of Improvement None None 3 Stakeholder Involvement Stakeholder Chart Completed Stakeholder Chart Stakeholder Chart Graphic 4 PQI infrastructure PQI infrastructure Graphic None 5 Model of Change Model of Change Graphic None 6 Improvement Plans Improvement Plan Completed Improvement Plan Example Plan (page 1) Do (page 2) Check and Act (page 3) 7 Program Indicators Program Indicators Worksheet Completed Program Indicators Worksheets 8 Logic Models Logic Model Goal Completed Logic Model example #1 for Programs Logic Model Actual Completed Logic Model example #2 for Administration 9 PQI Plan None See below 10 Sample PQI Plan Complete Infrastructure Graphic Sample PQI Plan 11 PQI Report None Sample PQI Report 12 Closing the Loop None None Council on Accreditation www.COAnet.org 1 Version 1.02; Revised November 21, 2016 13 Conclusion & Next Steps None None 14 Additional Guidance PQI Evidence Worksheet Mental Health Evidence Example Tracking Worksheet Human Resources Evidence Example Financial Counseling Evidence Example Tracking Worksheet Example 15 Bibliography None None 16 Print Version Download This PDF document includes all the resources listed above in the table of contents. The resources are organized into section within the PDF document. This allows users to print all resources at once instead of accessing the individual links above. (total of 127 pages; 12 MB) Council on Accreditation www.COAnet.org 2 Version 1.02; Revised November 21, 2016 1. Introduction Performance and Quality Improvement Tool Kit Welcome! This tool kit has been developed to assist organizations with implementing a comprehensive Performance and Quality Improvement (PQI) program. The tools that are included in this document are only intended to be guides. The Council on Accreditation does not expect organizations to use them in order to successfully achieve accreditation. However, the tools - when used appropriately and thoughtfully - will lead an organization, agency or program to a successful PQI system. The tools can be used exactly as they are stated, or they can be recreated to capture elements of your organization’s culture and climate. A strong Performance and Quality Improvement Program is intended to improve services to clients. COA’s PQI Standards expect that organizations will always be finding ways to improve their client care, regardless of how successful they currently are. A few notes about this guide: • The recommendations or guidance should not be interpreted as extensions of COA’s Performance and Quality Improvement Standards. For direct guidance on what is expected for successful TIP: accreditation in the area of PQI, please see the PQI If you are new to developing a PQI system, start Standards section for your relative Standards with the tools as-is. As you begin implementing Edition. the program, you will develop ways that work • The terms “organization,” “program,” and “agency” better for your organization. Those ideas are typically used interchangeably unless otherwise hopefully take the form of new tools that you noted. will create specifically for your organization. • The terms “PQI program” and “PQI system” are used interchangeably throughout this guide. Both terms are used to represent the activities, infrastructure, practices, procedures and other elements that relate to performance and quality improvement. • Use of these tools and resources are not a guarantee of accreditation. These resources simply assist organizations with documentation of implementation. While documentation is important, COA accreditation relies upon actual practice. For example, if your organization is utilizing all of the forms in this document, but is not conducting the activities, it will not succeed in achieving accreditation. • The use of this guide is not restricted to organizations seeking accreditation or currently accredited by COA. • Use of this guide is appropriate for large and small organizations. Organizations will have to contextualize the material to their specific needs. Council on Accreditation www.COAnet.org 3 Version 1.02; Revised November 21, 2016 2. Culture of Improvement Performance and Quality Improvement Tool Kit Defining Culture of Improvement Defining culture – regardless of context – can be a challenging feat. So how does COA define a culture of improvement? Looking at PQI 1 for the Private Edition Standards, we have some guidance (see below). Understanding that culture is unique to each organization, let’s take a broader look at what culture means and then relate it to improvement. According to The Social Work Dictionary, 6th Edition 1, culture is “the customs, habits, skills, technology, arts, values, ideology, science, and religious and political behavior of a group of people in a specific time period.” The group of people referring to the culture is easy: the organization and its stakeholders. To define the time period, we can describe that as present, with lessons from the past and plans for the future. Religious and political behavior may not be appropriate for this exercise, however, organizations can view themselves from the perspective of customs, habits, technology, arts, values, ideology and science. Some of the questions listed below, relating to the definition of culture, may be helpful in assessing if your organization has a culture of improvement. 1. What regular activities do you conduct that provide an opportunity for improvement? (habits and customs) 2. What talents do your staff members possess that contribute to improving practices? (skills) • Do you have staff who are skilled in staff development and morale building? • Are there team members who enjoy evaluation and planning? 3. What systems do you use to foster improvement? (technology) • What programs or software contribute to the organizations ability to measure, track, and aggregate data? 4. How do you use and encourage creativity to improve the services that your clients receive? (arts) • Are there creative means to recognize and encourage contributions to your organization? 1 Barker, R. L. (2003). The social work dictionary (6th ed.). Washington, DC: NASW Press. Council on Accreditation www.COAnet.org 4 Version 1.02; Revised November 21, 2016 • Do staff feel supported to mention ideas, even if they seem radical? 5. As an organization, how do your values relate to improvement? (values) • Does your mission statement or vision statement relate to improvement? • Is your organization willing to invest in people or ideas that may not have a direct monetary return? 6. How would you describe your organization’s model of change? (ideology) • Does your organization follow a Plan-Do-Check-Act cycle, Six Sigma, or other model of change? 7. What evidence do you use to demonstrate and support improvement? (science) • Are changes based upon evidence or gut feelings? • Are interventions provided because they are proven effective or because they “feel good”? As demonstrated above, defining a culture of improvement for any organization is not straight forward and can often be contextual. There are no “right” answers for the questions above; just an opportunity for critical thinking about how your organization values change. For that same reason, there is no tool or guide that will objectively change the culture of your organization. However, exploring the above questions with your leadership team and other key stakeholders of your organization may provide some insight. Council on Accreditation www.COAnet.org 5 Version 1.02; Revised November 21, 2016 3. Stakeholder Involvement Performance and Quality Improvement Tool Kit Identifying Your Stakeholders The first step to developing your organization’s PQI system is understanding who your stakeholders are and what role they play within your organization as well as in the improvement process. Below are steps to incorporating stakeholders into your PQI Plan and using the Stakeholder Chart Tool. Step 1: Make a list Organizations must identify who their stakeholders are. The list below will look familiar for most organizations. Some additional stakeholder groups that are not included in this list may be student interns, foster parents, government officials, military commanders, or law enforcement. You are encouraged to think critically and creatively about groups of individuals who have a stake in your organization and how to include them in your PQI system. Council on Accreditation www.COAnet.org 6 Version 1.02; Revised November 21, 2016 Step 2: Categorize your list Using the Stakeholder Chart Tool, indicate each category of stakeholder that you would like to include in your PQI system. List them in the “Stakeholder Group” box. You can be as detailed as necessary for the reader to gain a full understanding of who has a stake in your organization’s practices. Step 3: Describe each group Once you have outlined each stakeholder group, it is important to briefly describe that group and how they are unique to your organization. For example, all organizations are going to identify “clients” as a stakeholder group. What makes your clients unique? What services do they receive? Perhaps the clients at your organization receive mental health services and typically come from lower socioeconomic households. Or, maybe your organization works with many individuals who have recently been incarcerated and are struggling to find stability in society. They may not be identified as clients at all, and your organization uses the term “consumer” or “customer.” Look at the example below and note that the description is very brief, yet provides the reader with some context. Notice the feedback loop. Data comes from clients and then the information is given back in the form of improvement, summaries and reports. Step 4: What data does the stakeholder provider? To further solidify the readers understanding of how the stakeholder is involved in the improvement process, the next step involves documenting what data the stakeholder provides to the organization. Every stakeholder should be providing some type of data to the organization whether formal or informal. In the same example above, you will notice that the clients in this example provide satisfaction data (formal) through an annual survey, as well as comment cards that are available in the common areas. Clients can even provide informal data through organization employees by simply telling them about how things can improve. When the data is informal, it’s the responsibility of the recipient of the data (the employee) to communicate the data to appropriate staff at the organization. Council on Accreditation www.COAnet.org 7 Version 1.02; Revised November 21, 2016 Step 5: What information do you provide to the stakeholder group? Step five of involving stakeholders in the PQI system is “closing the loop.” This theme will be repeated throughout the PQI Tool Kit because it is an important concept and an area where many organizations struggle. Section 12 discusses the concept in greater detail. Closing the loop involves follow-through and follow-up. It reinforces buy-in and ensures that organizations do not simply collect data, but they actually do something with it. For this context, closing the loop means providing information back to the stakeholder. The final box that is included on the Stakeholder Chart asks for how the organization returns information to the stakeholder, as it relates to the data that they initially provided. For example, if clients regularly complete satisfaction surveys, but results are never shared, they may think that it is not worth their time to complete the surveys. By closing the loop and providing clients with a summarized version of the survey results, clients may be more apt to think that the organization is doing something with the data. Furthermore, if an organization shares with clients how they made a change based upon the feedback, clients may feel encouraged to be a part of the organization. In the example above, the organization returns information to the clients by making sure that the PQI Quarterly Report is available to clients and includes information on the areas of measurement that the clients participated in. The Stakeholder Chart is a simple document to help organizations gather their thoughts and practices about stakeholder involvement in one place. Descriptions are limited TIP: to simple summaries; the actual evidence would be visible in It might be helpful to conduct this as a copies of reports or correspondence that is provided to brainstorming activity. This will ensure that you stakeholders. do not leave any group out. Once they are all identified, they can be categorized, as appropriate, then placed in the Stakeholder Chart. Council on Accreditation www.COAnet.org 8 Version 1.02; Revised November 21, 2016 4. PQI Infrastructure Performance and Quality Improvement Tool Kit Establishing and Maintaining the Capacity to Have a PQI system Every PQI infrastructure is different. It is up to the organization to develop a system that is appropriate to the organization size, availability of resources, and complexity of services. However, there are common elements that make up all successful PQI programs. Step 1: Identify a coordinator The Coordinator can be defined as a point person who is responsible for overseeing the PQI process, organizing efforts and closing open PQI loops (see Section 12) throughout the TIP: organization. Some organizations may have a full-time person Don’t forget to include PQI responsibilities in job solely dedicated to this role. Others may have a fraction of a descriptions for appropriate individuals. full time person manage the responsibilities of the PQI system. Specifically, if you have a full time employee only There are also organizations that have an entire department, with only a portion of their time dedicated to with a Director. Regardless, this person or persons initially PQI. You will want to reflect those duties in their receives the data from stakeholders in the form of surveys, job description. You may want to even consider monthly reports, outcomes, dollar amounts, budgets, adding some duties to the job descriptions of spreadsheets and many countless other mediums. those who serve on the PQI Committee. The Coordinator is typically the collector, sorter and processor of data for the organization. Using existing systems, whether advanced software or simple spreadsheets and databases, the Coordinator has the capacity to organize the information and move it to the next component of the PQI infrastructure in a manner that is accurate, understandable and digestible. Step 2: Develop a PQI committee The PQI Committee works in close collaboration with the PQI Coordinator. Often chaired by the PQI Coordinator, the PQI Committee is a group of employees within the organization from all levels. The Committee has the responsibility of reviewing the information that is provided by the Coordinator from their unique perspective in the organization. For example, direct service staff members are able to view the data and understand how the data relates to the work that they conduct with their clients. The leadership of the organization may have a more macro-level view of the information. Administrative staff are often able to provide an angle on the data that others may simply look past. All perspectives are valuable and provide an opportunity to improve. Some common objectives of PQI Committees include the following: • Review data received across the organization. Council on Accreditation www.COAnet.org 9 Version 1.02; Revised November 21, 2016 • Identify and analyze trends in the data. • Brainstorm opportunities for improvement and change. • Recognize staff and stakeholders who have made contributions to positive change in the organization. • Generate excitement and energy around the PQI system in the organization. • Produce reports or dashboards that display the information that they have collected. • Develop improvement plans to address areas of concern or for general projects that support the work. Although PQI committees are typically comprised of staff members, some organizations have been successful at including other stakeholders, such as volunteers, clients, and board members. These organizations demonstrate an ability and willingness to go above and beyond in establishing a culture that values change. Step 3: Define the communication with leadership Organizations already have leadership, but it’s the relationship to the PQI infrastructure that matters in this context. It is expected that the PQI Coordinator has close communication with the leadership group, if the coordinator is not already a member of the leadership at the organization. The leadership’s role within the organization is to support PQI initiatives by: • Allocating appropriate resources to the PQI system. • Carefully considering requests and improvement plans that are developed within the PQI Committee. • Participating in PQI activities, as appropriate. • Supporting and encouraging a culture of excellence and continual improvement. • Inviting and involving external stakeholders in the improvement process, as appropriate. It often helps to provide a diagram or infographic of the PQI structure to help readers understand how information flows. On the following page is an example that organizations can use or adapt to meet their specific needs. This graphic is also available for download. To walk through this graphic, let’s start with the stakeholders on the left side. They provide the data, which you will notice is on the right side of the stakeholders with arrows pointing to the right. The data they provide is collected, reviewed and aggregated by the PQI Coordinator. For large organizations, a team will be completing these tasks. For small organizations, the director of a program or an administrative staff member may be completing these tasks. Once the data is in a more digestible format, it is delivered to the PQI Committee. You’ll see a red arrow that points down, to the committee. There the committee processes the information, identifies trends, areas of concern and improvement plans. That information is provided back to the PQI Coordinator, as demonstrated by another red arrow that points up. This exchange of information may happen multiple times or on an ongoing basis. That same back-and-forth may also occur with the leadership of the organization and the PQI Coordinator. The leadership will want to know the trends, concerns, and improvement plans to provide approval or to just have an awareness. Whatever happens with the data, whether it’s just placed in a report or is the catalyst for an improvement plan, the results and progress are given back to the stakeholders. This flow is demonstrated by the large curved arrow pointing from the PQI Coordinator to the stakeholders on the left. The process is typically ongoing. Council on Accreditation www.COAnet.org 10 Version 1.02; Revised November 21, 2016 This diagram is explained on the previous page. It is intended to serve as a model for organizations and can be copied or altered as needed. If copied, be sure to contextualize the model to fit your organization’s specific needs. Identify the flow of information, the roles, committee structure, reporting mechanisms, etc. Council on Accreditation www.COAnet.org 11 Version 1.02; Revised November 21, 2016 5. Model of Change Performance and Quality Improvement Tool Kit Describing Your Model of Change When reviewing the various data that is processed through the PQI infrastructure, there will be trends that are identified as needing to change. To demonstrate that your organization implements change in a thoughtful manner, it is necessary to identify a model of change that PQI initiatives are implemented through. Remember, not all data or information needs to go through a model of change. Only data that indicates the need for change requires the use of a model of change. Step 1: Identify a model of change that works for your organization A common model of change that many organizations use, both formally and informally, is the Plan-Do-Check-Act (PDCA) cycle. Some organizations may recognize this by different names such as Plan-Do-Study-Act, Plan-Do-Check-Adjust or the Deming Circle. Regardless of the name used, the concept is the same. It is an organized way to view and structure improvement initiatives. We will provide additional information about the use of this model further in this section. Also, it is important to be aware of other models that might be a better fit for your organization. Below are some examples and links to where you can find additional information: (The hyperlinks are to Mind Tools Inc. There are many resources available online for additional models of change for organizations. 2 The Council on Accreditation does not endorse Mind Tools Inc., nor do they have a financial or business arrangement). • Lewin’s Change Management Model (more information available on www.mindtools.com) • McKinsey 7-S Model (more information available on www.mindtools.com) • Kotter’s 8-Step Model (more information available on www.mindtools.com) • Six Sigma (more information available on www.mindtools.com) Step 2: Identify when change is needed • Every organization will have different ideas about what requires change. As stated in the beginning of this section, not everything that is reported by stakeholders is in need of change; in fact, most information will likely get aggregated and reported back with some additional context. For example, when monthly numbers are collected for Program A, do those numbers need to go through a model of change? No, unless they identify a trend that needs to be addressed. Perhaps they have decreased over the past 4 months, or they have remained the same even though a new staff member has been added to the team. It is the PQI Committee and the PQI Coordinator’s responsibility to identify when the data is actually information that requires change. Otherwise, the information will be part of a greater report and returned to the stakeholder groups with no action taken. 2 Mind Tools: Online Management, Leadership and Career Training. (n.d.). Retrieved September 16, 2016, from https://www.mindtools.com/ Council on Accreditation www.COAnet.org 12 Version 1.02; Revised November 21, 2016 • When change is identified, that is when the model of change is applied. The following section will provide a method for implementing and documenting change using the PDCA model. Step 3: Implement your model of change For the purposes of this document, it will be assumed that you and your organization are selecting the Plan, Do, Check, Act Cycle. 3 4 It is very common and very simple. If your organization chooses another model, please keep that model in mind when PDCA is referenced. Implementing that model of change, specifically using the PDCA cycle often comes down to documentation and being more thoughtful in the change process. The first task is understanding the model and how its use will impact the organization. Most often, graphics are used to demonstrate how the model is used at organizations. The PDCA model is simple to demonstrate through the use of graphics, yet it is also flexible enough to adapt to a multitude of situations and contexts. Below, you will find a simple graphic that helps users visualize how the process works. In fact, many readers may see this cycle and recognize that this is already the process that they use to produce change. It’s intuitive, which is another reason for its common use. 3 Moen, R., & Norman, C. (2006, April). Evolution of the PDCA Cycle. Retrieved September 16, 2016, from http://pkpinc.com/files/NA01MoenNormanFullpaper.pdf 4 PDCA. (n.d.). Retrieved September 16, 2016, from https://en.wikipedia.org/wiki/PDCA Council on Accreditation www.COAnet.org 13 Version 1.02; Revised November 21, 2016 Of course, understanding the model of change goes deeper than looking at a graphic. As part of implementing a model of change comes the responsibility of educating organizational staff on its purpose and use. Below is a brief outline of each step in the cycle. PLAN During this phase of the Plan, Do, Check, Act Cycle, preparations are made in order to effectively make the change. This may involve gathering additional data and information to support the need for the change. If collaboration is required for the potential change, the plan phase may involve committee meetings and the development of proposals and work plans. If it is a smaller change, the planning may simply involve brainstorming about the possible implications to all aspects of the organization. Near the end of the planning phase, a work plan is developed to identify specific objectives, responsibilities and indicators of success. DO During the Do phase, the work plan or proposal is acted upon. There should be a mechanism in place for follow up and regular reporting on the status CHECK This phase allows for the work to be assessed, after an appropriate period of time has passed. Those involved review the process and identify the positive aspects of the change and any negative aspects of the change. The group or individual responsible for the change compares the actual results to the expected results. Deviation from the expected result is noted. An assessment of how the change impacts the rest of the organization should also take place. Most importantly, it needs to be determined whether the change was successful or unsuccessful. Even if unsuccessful, the change still has some positive impact, even if that impact is simply a better understanding of the situation. ACT Once the organization has determined if the change was an improvement from the baseline, or prior status, then that change is accepted as part of current practice. The organization maintains the new practice or change. The organization will need to integrate this change into their current culture and environment. If the change was not a positive one, then they may return to the baseline way of operating. The organization, group or individual can start another Plan phase of the PDCA cycle to determine a different method to address the concern. A detail that is easy to miss in the above model of change graphic is the uphill climb, also known as progress. Improvement cycles should always be occurring; organizations can benefit the most from a PQI system when there is always at least one improvement plan being currently implemented. As stated earlier, even when improvement plans do not work as expected, lessons are learned, the organization grows and progress ensues. As per the visual, when there are no improvement plans in progress, the wheel stops spinning and rolls back instead of moving forward. Council on Accreditation www.COAnet.org 14 Version 1.02; Revised November 21, 2016 6. Improvement Plans Performance and Quality Improvement Tool Kit When change is needed As discussed in the previous section, some information is indicative of change. With the Plan, Do, Check and Act cycle as the framework, this section will go through using improvement plans to implement the change. The links below are tools designed to guide organizations through improvement initiatives using the PDCA cycle and documenting it appropriately. Maintaining good records of improvement initiatives - whether successful or not - provides ideal demonstration that change happens and lessons are learned. These worksheets are designed to guide organizations in using improvement plans, consistent with COA’s PQI Standards, specifically PQI 7.01 (Private & Canadian Editions). Page One of the Improvement Plan: Plan Page Two of the Improvement Plan: Do Page Three of the Improvement Plan: Check & Act Step 1: Identify the need to implement an Improvement Plan Improvement plans can be used for a variety of reasons, including: 1. When data collected through an organization’s PQI program indicates that there is an area of concern. 2. For administrative functions that are in need of increased efficiencies. 3. To correct under-performing programs or sites. 4. To provide guidance to staff members who demonstrate performance that is not meeting expectations. 5. For successful programs that an organization would like to further develop. 6. To increase the involvement of the governing body. These are just some examples of how Improvement Plans can be used; however the number of opportunities for their use is unlimited. It is important to talk with staff and stakeholders about the role of Improvement Plans within the organization. While they can sometimes be used to document and improve undesirable behaviors or performance, they are not intended to be punitive. On the contrary, by programs/departments always having a working Improvement Plan in place, it is a demonstration that the organization has a culture of improvement and values change. By reviewing data collected through the PQI program, looking at surveys, talking with staff or meeting with members of the governing body, an organization should be able to identify something that can use improvement. It’s important to have data to support the need for change. If the data is not present, but an organization still recognizes a need for Council on Accreditation www.COAnet.org 15 Version 1.02; Revised November 21, 2016 change based on anecdotal information, gathering data to support the change can be part of the planning process. In summary, use data to identify and support the change. In PQI 7.01, the interpretation discusses the use of corrective action plans and Improvement Plans. Corrective action plans are used in response to an identified deficiency and are often more punitive. Improvement plans are more proactive in nature; however, for the sake of simplicity, an improvement plan can also be used as a corrective action plan. If this is the case, it will be important to identify the reason for the plan and the consequences of not successfully completing it. Step 2: Assign a responsible person or persons Now that the opportunity for improvement has been identified, the organization will identify who is responsible for developing the improvement plan and ensuring that the appropriate staff are on track with achieving the objectives as listed in the plan. For many organizations, the individual identified as the PQI Coordinator may be TIP: the responsible person for developing and tracking improvement plans. Start small! If this is a new process for Perhaps for other organizations, each Director and Manager is the organization, start with a simple responsible for creating plans for their respective team or program. Improvement Plan. When it is fully Either method is appropriate, as long as there is someone who develops followed-through and successful, it will the plans, maintains them, monitors them and provides follow-up when contribute to developing buy-in with necessary. staff who may have felt apprehensive. Also, it helps to not have the first Let’s turn to the first page of the Improvement Plan document. On the Improvement Plan as a result of the following page, a view of the form is available. need for corrective action. Use an opportunity that is proactive and has Step 3: Plan the support of the organization’s staff. Complete the Plan form of the Improvement Plan. On this form, you will be asked the following questions: 1. Briefly describe the opportunity for improvement and what information supports this need. 2. Describe the success indicators; how will you know that the proposed actions were effective? 3. What data supports the need for this change? Council on Accreditation www.COAnet.org 16 Version 1.02; Revised November 21, 2016 In the example above, notice that the situation is described in detail. It provides you with an overview of the opportunity for improvement and what the organization did to gather more data. Because this is an optional form – a tool – the exact language of what to include is up to you. Complete this information in a manner that will be beneficial not just to COA, but also to your organization and team. Step 4: Develop the work plan Complete page 2 of the Improvement Plan by TIP: detailing the specific action items of the initiative. While organizations that use this tool will most likely use it There are six separate categories that organizations to meet COA standards or to respond to COA’s Commission should complete: action item, person responsible, Department, COA wants this to be a helpful tool for cost and resources needed, target completion date, organizations, not COA. If adjustments are needed or if actual completion date and success organizations want to create a similar form, but tailored to indicators/comments. Organizations will find more their unique needs, they are encouraged to do so. Council on Accreditation www.COAnet.org 17 Version 1.02; Revised November 21, 2016 benefit if greater detail is included. Additional pages can always be added if necessary. Step 5: Complete the follow up Once the plan is documented and tasks are delegated to appropriate staff TIP: throughout the organization, the next step is to complete what was planned. If organizations are finding that they have too many action items, it might be helpful to divide the Improvement Plan into two or more Understanding that things change, it is Improvement Plans. Remember, the plans should be achievable and important that organizations monitor the time-limited. If a plan exceeds a target completion date of 6 months, progress on all Improvement Plans. perhaps it needs to be broken down into more achievable Organizations sometimes use PQI Improvement Plans. That, too, will help with buy-in. It always feels Committee meetings to receive updates good to close the loop on an Improvement Plan! on progress. Other organizations may use Management Team meetings or a Council on Accreditation www.COAnet.org 18 Version 1.02; Revised November 21, 2016 specific point person who has the capacity to follow up on the implementation of the plan. Below are some ideas for how to ensure that the plan is being utilized: • Improvement Plans can be part of meeting agendas to prompt follow up and reporting. • Brief monthly reports may be used to document the progress. • Supervision times can be used to review Improvement Plan progress. Steps 6: Reflect on the process and integrate the results into current practice, if appropriate At this point, the Improvement Plan should be complete and assessing the performance of the Improvement Plan takes precedent. The Organization should discuss/answer the following questions, as per page 3 of the Improvement Plan: 1. Describe the results and observations of the Improvement Plan. 2. What challenges were encountered during the implementation of the Improvement Plan? 3. Describe how the changes implemented through the Improvement Plan will be integrated into regular practice. Please remember that Improvement Plans that do not produce the intended results are not failures. Some Improvement Plans teach organizations what NOT to do. It is important to continue to document the results and learn and reflect on the process. The following page is an example of a completed form for the Check and Act phase of the PDCA cycle. TIP: Once the Improvement Plan is complete and the follow through has taken place, be sure to keep a record. Improvement Plans can be summarized in order to report them back to stakeholders, if appropriate. Also, it’s great to be able to reflect back on the changes that were made and changes that were not successful. These can be great learning tools for future organizational leaders! Council on Accreditation www.COAnet.org 19 Version 1.02; Revised November 21, 2016 Council on Accreditation www.COAnet.org 20 Version 1.02; Revised November 21, 2016 7. Program Indicators Performance and Quality Improvement Tool Kit Documenting What You Measure In developing your PQI system, it will be necessary to identify what you are measuring and how you are going to measure it. COA’s PQI Standards (Private and Canadian) outline what type of measurements need to be identified for each program and site. The Program Indicators Worksheet was designed to provide an organized way of identifying what those measurements are for each specific program that you may have. If using the Program Indicators Worksheet to demonstrate what indicators are being measured, you will need to complete a worksheet for each program that your organization has. Remember, COA Standards require that the PQI Plan at your organization covers all sites and programs. This even includes administration. Typically, administration is not as clear cut as programs and sites can be, but some examples are provided for how that may look. These worksheets can be included as part of your PQI Plan, or they can help you organize the information so you can place it in a format of your choosing. Step 1: Create a list Create a list of all of the different programs, projects and departments for which indicators exist or will be developed. For example, your list may look like this: • The Mental Health Program • Foster Care Program • Financial Counseling Program • Broadway Office Mental Health Program (Site Specific) • Finance Department (Administration) • Human Resources (Administration) You’ll notice a few things about this example list. Each program is initially broken out by the type of service. This tells the reader that those programs have measurements that are consistent across sites and facilities. For example, let’s say that the Foster Care Program has multiple sites across New York City. By listing it only once, it assumes that the outcomes and outputs that are measured are the same across all sites. You will also notice that the Mental Health Program is listed, but then the Broadway Office Mental Health Program is listed separately. That tells us that there is something specific about that program that needs to be separated out. Perhaps that location receives funding that is only to be used for that location and the funder needs information only related to that location. If that is the case, list the program separately. Secondly, let’s notice the administrative departments. COA’s PQI Standards require that organizations identify measures that apply to administrative operations of the organization, such as finance, human resources, risk prevention, and others. Some organizations that are smaller may not have separate departments or divisions within their administration. Perhaps the finance department also controls risk prevention activities as well as human resources. In Council on Accreditation www.COAnet.org 21 Version 1.02; Revised November 21, 2016 this case, the organization may choose to simply list the program/project/department as “Administration”. Larger organizations are going to have more divisions and consequently, have more areas of measurement. There will be more discussion about the lists of programs, projects and departments further on in this document. Step 2: Identify the outputs Using the Program Indicators Worksheet, linked here, identify the program name, project name, or operation name at the top then move to the section titled “Outputs.” In the outputs section, this first box is an opportunity to list the different outputs that are measured for the relative program. Outputs are typically numerical values that represent an amount of the program provided or deliverable. For example, outputs can include all of the following items: • Number of hours of service provided. • Units of service. • Length of time from initial contact to intake assessment. • Number of new clients for a specified period of time. • Number of successful terminations. • Turnover rate. • Staff retention rate. • Revenue generated. You will notice that all of the outputs above are the result of whatever intervention is being provided or action taken, but it does not necessarily mean that any positive change has taken place. If an organization provided a total of 1,300 units of service during the month of August, it tells you that they worked hard, but not that the intervention was successful. To look at an administrative example, revenue generated does not necessarily indicate the overall health of the organization. Perhaps an organization generated a great amount of revenue, however, the expenses that accompanied the revenue were greater. Many organizations collect numerous outputs; more than what the space on the Program Indicators Worksheet allows for. Consider the outputs that are collected that are actually tracked, measured and benchmarked. Those are the outputs that should be listed. Let’s say that an organization’s mental health program has mechanisms in place to collect number of clients, average time between initial contact and first appointment, average length of time in treatment and average number of sessions. All of these outputs can potentially be beneficial, however the organization is only concerned with the number of clients and average number of sessions. The other outputs are collected, but nothing is Council on Accreditation www.COAnet.org 22 Version 1.02; Revised November 21, 2016 done with the data. If this is the case for your organization, then only those outputs for which the outputs are actually used should be listed. Step 3: Identify the outcomes After identifying the outputs, the outcomes will now need to be identified. This can often be the most challenging part of developing the PQI system. Having a strong outcomes statement is crucial in measuring the success of your program. For more information on developing a strong outcomes statement, view the logic model section of the PQI Tool Kit. Remember that COA Standards require that each program measures at least two of the following: • Change in clinical status • Change in functional status • Health, welfare and safety • Permanency of life situation • Quality of life • Achievement of individual service goals The Items Measured box provides a great opportunity for you to highlight which of these categories are being measured. Once those outcome items have been identified, you can then go into more detail in the Details box. This is an opportunity to include what measurement tool is being used and at what frequency. It is recommended that you don’t use your actual outcome statement in this section because that may change from year to year. The specific outcome statement will be documented in your Annual Plan or logic model. Remember the Program Indicators Worksheet is intended to serve as an outline for what you are measuring and how; it does not need to be the actual goals and objectives of the program. The logic model or Annual Plan will have specific goals and objectives. The final area in the outcomes section refers to documentation. As with outputs, this box allows you to briefly state how the outcomes are reported back to stakeholders. As part of closing the loop, you can indicate things such as: • Outcomes are reported back to stakeholders on the PQI Quarterly Report. • The organization produces a dashboard on its website that contains live information and is updated as data is entered. Council on Accreditation www.COAnet.org 23 Version 1.02; Revised November 21, 2016 Step 4: Review the quality indicators The Quality Section of the Program Indicators Worksheet gives organizations an opportunity to document what measurements are included to ensure that the services are provided with integrity and professionalism. Items measured can include some of the following examples: • Chart review • Client satisfaction • External reviews and audits • Utilization review • Adherence to the treatment model; fidelity As with the previous categories, details can be provided in the appropriate box and the documentation box will inform the reader of how the quality indicators are reported. Step 5: Measure program-specific administration performance Admin. or administration is the section where organization can discuss what items are reviewed on an annual basis to ensure that they are efficient and effective for the delivery of service. COA’s standards require that at least annually, the organization assesses functions of their programs. This provides organizations with the ability to look at procedures that “have always been in place” and ask themselves if they are the most appropriate procedures to have in place. Is there a better way? Can we improve how we conduct our intake assessments? Are our time frames too long? These are just some of the examples of questions that organizations can ask themselves during this annual review. In the items measured section, you will see some generic areas are assessed. While there are not formal tools typically to measure these items, the details section can document how the organization does it. Perhaps it is a simple discussion that happens with the team followed by a brainstorming session. Perhaps an improvement plan will result when an area is identified that can be improved or changed. Typically, for this area of measurement, the only documentation that will be present are minutes from the meeting in which the discussion occurred, or perhaps that completed improvement plan has the information needed. Step 6: The Program Indicators Worksheet is complete! One should be completed for each program. They can then be incorporated into the PQI Plan as evidence of what is measured, how often and using what tool. If you choose, you can Council on Accreditation www.COAnet.org 24 Version 1.02; Revised November 21, 2016 transfer this information into an Excel spreadsheet. This may be more practical for large organizations that have many programs and administrative departments/functions. Having one page per program/department can result in a huge stack of paperwork! TIP: The intention of this form is to make sure that you have the appropriate measurements for each program. If you have a program documented on a Program Indicators Worksheet, but there are incomplete areas, it most likely means that you are not implementing PQI Standards fully. Allow this document to serve as a checks and balances for each of your programs. Council on Accreditation www.COAnet.org 25 Version 1.02; Revised November 21, 2016 8. Logic Models Performance and Quality Improvement Tool Kit Using Logic Models The logic model worksheet is one of many types of logic models that can be used to demonstrate and document an organization’s achievement towards goals. The goals discussed and recommended utilize an outcomes perspective – they are focused on achieving a goal that is sustainable, narrowly-focused, time-limited and measurable. The tools can be located here: Logic Model Goal and Logic Model Actual. PQI 4, Performance and Outcome Measures, references the use of logic models, however does not require it. Using logic models for this section makes sense and simplifies the intent of the standard. Remember, organizations can choose to use these as they see fit, however many funders and stakeholders require that organizations use logic models. Organizations that already have to complete logic models as a condition of their funding contract can simply use the same model as part of the PQI system. There is no need to replicate work that is already taking place. The development of these tools has been supported by two specific publications. The W.K.Kellogg Foundation’s Logic Model Development Guide 5 is a great resource for training staff and becoming acquainted with the logic model framework. Also, The Nonprofit Outcomes Toolbox: A complete guide to program effectiveness, performance measurement and results 6 is another wonderful resource for organizations to use in developing and maintaining a strong performance and quality improvement program. Step 1: Identifying and developing the outcomes statement Since most organizations will be using logic models for current and existing programs, the remainder of the instructions will be focused to that perspective. To begin this process, organizations need to understand what makes a strong outcomes statement. Part of understanding what makes a strong outcomes statement is understanding the difference between outcomes and outputs. These two terms are often confused. There are many resources available on line, as well as the two resources mentioned above, that can clearly define the difference. To briefly summarize, outputs are the results of the activities that may not necessarily equate to positive change. For example, if a foster care worker spends 25 hours of direct service working with resource families, does that mean that the children in care have safer homes? Not necessarily. To find out if children are in safe and secure family environments, a specific measurement tool or treatment plan goal would need to be used. Outputs are typically a count of hours, dollars, clients, units of service, etc. They are the direct results of activities that were conducted but they are not indicative of change. If someone is unsure about whether something is an output or an outcome, they can ask themselves this question: “Based upon the information in review, is it possible that no change took place?” If the answer is yes, then the indicator in question is most likely an output. 5 W.K. Kellogg Foundation evaluation handbook ; W.K. Kellogg Foundation logic model development guide. (2005). Battle Creek, MI: W.K. Kellogg Foundation. 6 Penna, R. M. (2011). The nonprofit outcomes toolbox: A complete guide to program effectiveness, performance measurement, and results. Hoboken, NJ: John Wiley & Sons. Council on Accreditation www.COAnet.org 26 Version 1.02; Revised November 21, 2016 Programs often times only have two or three outcomes. That is typically sufficient. Numerous outcomes can be challenging and complex to consistently measure and track. 1. Outcome statements should have a perspective that is narrow and specific. 2. Outcome statements should be objective and measureable. 3. Outcome statements should indicate some level of sustainability. In addition to the guidelines above, outcome statements should measure one of the following aspects of the service recipients: B • Behavior A • Attitude C • Condition K • Knowledge S • Status B: Behavior. Example: A smoking cessation program would measure a change in behavior from smoking 1 pack a day, to smoking 0 packs per day or perhaps a lesser amount such as 1 pack of cigarettes per week. The focus is the behavior of smoking. A: Attitude. Example: A men’s batterer intervention program may seek to change the perpetrators’ attitude towards women (for heterosexual relationships). C: Condition. Example: A mental health program may seek to change the client’s condition of moderately depressed to mildly depressed or not depressed. K: Knowledge. Example: A credit counseling agency may seek to improve the service recipient’s knowledge of budgeting. S: Status. Example: A foster care organization may seek to change the status of the children that they serve from an unsafe family environment to a safe and stable family environment. Notice the use of the acronym BACKS: behavior, attitude, condition, knowledge and status. It is an easy way to remember what outcomes need to be measuring. Council on Accreditation www.COAnet.org 27 Version 1.02; Revised November 21, 2016 Many organizations have outcomes statements already developed for them by their respective funding source. If this is the case, that outcome statement(s) would be appropriate to place in the logic model, as long as it is actually an outcomes statement. On occasion, the funder’s expected outcome may actually be an output or an unmeasurable statement. If this is the case, organizations should still include it, but also add an additional goal that truly represents a strong, measurable outcomes statement. As the remainder of the logic model is developed, it is common that the outcomes need to be revised. Developing a logic model is a process and that process should be reflected upon and recognized as a positive experience for those involved. Step 2: List the inputs The next step in completing the logic model is including the inputs that are necessary to produce the change that the organization is hoping to achieve. Inputs refer to the tangible and intangible resources required to provide the intervention. For example, the first and foremost resource that all programs need is money. You may choose to indicate a specific dollar amount that is required to appropriately fund the program. Using an example, let’s say that an organization receives a grant for $320,000 for a program, but it actually costs the organization a total of $400,000 to implement the program. That organization might indicate the specific amounts to demonstrate to their funders that additional funds are needed to support the expected outcomes. This can be an effective way to show the need for additional funding from another source for the purposes of leveraging the program. Other inputs that organizations often use in the inputs category are staff, staff training, supervision, supplies, computers, office space, use of a vehicle, staff transportation, etc. Organizations are encouraged to use this category in a way that makes sense to them. Step 3: Document the activities and interventions Now that the organization is aware of the outcomes that they are hoping to produce and what resources are going to be required, it is time to document the activities or interventions that will lead to the outcomes. Being thorough with the activities can be helpful because it shows the reader how much work actually takes place to achieve client success. Typically, the main activity that organizations would want to document is the intervention that is occurring. For example, if there is a foster care program completing the activities section, they should include activities such as: training for potential foster care parents, strengths-based case management, quarterly educational seminars for foster care parents, intake assessments, treatment planning, etc. Notice that included in the activities list is the actual intervention, but also activities that lead up to the intervention. Most likely, the outcome for the foster care program will be related to achieving permanency for the child. Activities leading up to that intervention are numerous but they are all part of the service provided. This can get confusing at times. Let’s take a look at the example of the foster care program again. Part of the service is training for the foster care parents. That would be included as an activity. But what about the training for staff to provide the services? That type of training would typically be listed as an input. Staff receiving training is not part of the actual service provided. It is necessary, however, to train staff to appropriately do the work. Staff training requires resources and infrastructure, which is why it goes into the inputs category. The graphic on the following page demonstrates some of the different items that are appropriate to document in the activities section. Council on Accreditation www.COAnet.org 28 Version 1.02; Revised November 21, 2016 Trauma- informed case Cognitive management Educational behavioral seminars therapy Treatment Crisis plans counseling Monthly Psychosocial phone call assessments check-ins Intake assessments Activities Reassessments If you are uncertain to what category an action or item should fall in, the following questions may help provide guidance: • Is it part of the service provided? o If yes, then it is most likely an activity. o If no, then it is most likely an input. • Is it something that the client participates in? o If yes, then it is most likely an activity. o If no, then it is most likely an input. Step 4: Identify the outputs that you will be measuring Now it is time to define and identify potential outputs. To define outputs, it can be helpful to start with a definition that comes from economics. Outputs are the number of goods and services. For human service organizations, it’s the number of services that the activities produce. It’s important to take notice that the definition does not speak to the quality of those services or whether the services resulted in positive change for the recipients. Some common examples of outputs include the number of counseling sessions, hours of direct service, number of billable hours, number of education sessions, number of service recipients, etc. Regardless of how the outputs of a specific program or organization look, these numbers do not indicate the actual impact of the service. For example, if an organization provides a total of 150 units of service in one week for a mental health program, it does not necessarily mean that the clients are feeling better; it simply means that clients were in attendance during the week. Outputs often relate to productivity or compliance. Although outputs are not indicative of outcomes, they are still very important. Many organizations are paid for services provided based on outputs. Often times, organizations are able to bill for unit of service as opposed to billing based upon how much a client improves in functional status. Council on Accreditation www.COAnet.org 29 Version 1.02; Revised November 21, 2016 Step 5: Identify the need and the intended impact At this stage, the basics of the logic model are complete. The inputs that are needed to conduct the activities are identified. The activities and interventions that are going to produce the outputs and outcomes are listed. Outputs have been documented to demonstrate productivity and compliance. Finally, outcomes or statements of expected client impact have been developed and possibly refined. The next steps provide a context for the reader. The problem statement briefly describes why the program is in existence. This can be a brief statement that references statistics or research for the particular community that the program services. A problem statement helps the reader understand why there is a need for the program. Below are some examples: • Domestic violence has increased by 12% in our community during 2015 and resulted in 2 fatalities. Based upon a needs assessment conducted by Group B in 2015, services for victims of domestic violence are lacking and do not reflect the significance of the problem in our area. • On average, there are 1,500 children in foster care at any given time, however, there are currently only 1300 resource families in the area, which results in multiple children in one home and long term stays at short term residential facilities. The impact may be a general statement about how the program hopes to affect the community. Impact statements do not always have to be measureable; it is up the organization. There are many examples of where impact statements may not be realistic for an organization to measure. Below are some specific examples of impact statements, both measurable and not measureable. • The foster care program will place and maintain children in safe, nurturing and loving homes. (Not measureable) • The Credit Counseling Program will help consumers to be independent and free from debt. (Not measureable) • One year after case closing, 75% of successfully completing parents will continue to use the skills they learned in the parent education class. (Measureable) • Two years after successful discharge, 60% of clients will still be free from using substances. (Measurable) Sometimes the impact statements are divided up into different time frames, such as short term, medium term and long term. Impact statements provide the viewer with a general idea of how the program benefits the community and is a return on investment for the funder. Step 6: Use the logic model to track progress The logic model is complete at this stage; however, there is an option to use the completed logic model as the means for tracking progress and including indicators of success. Moving to the outputs column of the logic model, this is an opportunity to list what the organization expects of the program. Programs or organizations can include the actual and target indicators of success in this section. See below: • By December 31, 2017, Program A will provide a total of 1500 intake assessments to interested clients. • By December 31, 2017, Program A will complete 100 treatment plans. Notice that timeframes are indicated, as well as specific numbers. For the first example, it can be assumed that one of the program’s activities is providing intake assessments, as this is part of providing the service. By placing a number in this section, a target or goal has been identified. This can also be used to for tracking purposes as the year progresses. If every program at an organization completes a logic model in the manner described above, the data can be easily Council on Accreditation www.COAnet.org 30 Version 1.02; Revised November 21, 2016 placed in a graph and tracked. Below is a sample from a completed logic model for how this can look. The highlighted areas indicate where changes have been made to report on outputs. Notice how the goals are built right into the logic model. No need to create a separate document. Again, notice how the actual vs. goal is built right into the logic model. Goals Actuals For organizations to make best use of this model, the “Actuals” logic model can be updated on a quarterly basis. If the organization desires to have information presented in a chart or graph, it is easy to pull the data to display. Organizations and programs should remember that this model is just one way of many possibilities to demonstrate implementation of COA’s PQI Standards. For two examples of a completed logic model, click here for a program-specific logic model and click here for an administrative example. Council on Accreditation www.COAnet.org 31 Version 1.02; Revised November 21, 2016 As with the outputs, outcomes can be measured in a similar manner, using the logic model as the recording document with actuals compared against goals. While reviewing the outcome statement listed to the left, a few aspects can be noticed: • There is a time limit. • An evidence-based measurement tool is used. • Specific, measurable indicators are outlined. • The outcome measures condition of the client. • There is transparency in how the scores are compared, as it states “self-report” using the indicated measurement tool. Step 7: Implement organization-wide Once the organization or program has a good understanding of how to complete the logic model, develop outcomes and identify appropriate outputs, the model can be applied to each program. The logic models can serve as the “Annual Plan” that COA recognizes in the PQI Standards. COA understands that organizations must complete many reports and plans. We encourage you to consolidate and use formats that can fulfill multiple purposes. Other ways to use logic models and miscellaneous information: The examples discussed in this guide are focused on programmatic or client service areas of organizations. However, logic models can also be used to outline goals and actual progress for administration and management of the organization. COA requires that organizations include administrative goals in their PQI system. In constructing an Annual Plan, the logic model can serve to outline the delivery of client services for each program and/or site for the organization as well as areas such as human resources, finance and management efficiencies. Organizations are encouraged to include direct service staff, when possible, in the development of logic models in order to increase involvement and ownership of the process. The direct service staff, often times the experts of the work, will have great ideas and feedback in goal setting. Management will have the challenge of balancing goals from staff, funders, and leadership into a comprehensive and well-organized set of logic models. Council on Accreditation www.COAnet.org 32 Version 1.02; Revised November 21, 2016 9. PQI Plan Performance and Quality Improvement Tool Kit Putting the Pieces Together Putting together the PQI Plan is essential to having a strong foundation for a PQI system. Often times there is confusion between the Annual Plan and the PQI Plan. The intent of this section of the PQI Tool Kit is to demonstrate what should be included in a solid PQI Plan and how it is related to an Annual Plan. The overarching goal of the PQI Plan is to outline the organization’s approach to ensuring that programs and operations are always looking for ways to improve by reviewing keys sources of data. The PQI Plan does not say what the goals of the organization or the programs are; the PQI Plan states how the goals are developed, what tools are used to track the achievement and how reviews within the organization take place. The PQI Plan is like a blueprint; it tells you where and how to construct. The actual construction takes place separately. Different contractors and specialists provide services such as concrete pouring, electrical, framing, and plumbing. They reference the blueprint, but have specific plans that speak to the work that they are doing. The blueprint shows the big picture and guides the specialized work, but a home cannot be built off of the blueprint alone. The Annual Plan is where organizations will be identifying what goals and objective they have. Often times, an Annual Plan is simply a collection of all of the programs and operations individual “work plans” or logic models that document what they are going to accomplish that year. Many of the Annual Plans are related directly to the funder because the goals and objectives are tied to contract requirements. Annual Plans must meet the needs of the organization and still demonstrate that the organization is working towards achieving successful client outcomes, effective organizational operations and strong financial sustainability. If we use the same analogy as above, the annual plan can be compared to the specific work orders for the specialists. The plumber will have specific plans, as well as the electrician and the workers building the frame. The Annual Plan can be viewed as the operational, day-to-day work that needs to take place. The Strategic Plan has a broader reach and is often focused on ensuring that the organization is moving forward, accountable to the community it serves and consistent with the mission. While these themes are present in the Annual Plan, the Strategic Plan is more explicit in carrying out these objectives. Strategic plans should be for 3-5 years and developed by the Board of Directors, organizational leadership and select staff. On the following page are some examples of goals that may appear on both plans, but demonstrate how the Annual Plan is more operational and specific and the Strategic Plan is broader and focused on organizational leadership and vision. Council on Accreditation www.COAnet.org 33 Version 1.02; Revised November 21, 2016 Annual Plan Goal Strategic Plan Goal The Independent Living Skills Program will, on average, The Organization will support homeless youth by increase the client’s level of functioning by 15% when implementing a new Independent Living Skills program, comparing the pretest to the post test conducted at funded through the State of New York. termination according to the Self-Sufficiency Matrix. The organization will provide a supportive work The organization will increase staff retention by 5% as environment by improving, on average, 10% on the staff compared to the previous year’s staff retention rate. satisfaction survey this year as compared to last year. Many of the components included in this document can go directly into your PQI Plan. If you utilized all of the forms that have been created, they can be attached as addendums or embedded as part of the plan. Depending on your organization and the needs of the PQI Plan, how information is presented is up to you. Also included in this section of the Tool Kit are references to specific standards. This will allow you to review the actual source of the information and gain a better understanding of how PQI fits into the bigger picture. At the end, you will find a link to a completed PQI Plan for a fictitious organization that can be used as a guide in the development of your own plan. Recommended Outline of a Performance and Quality Improvement Plan I. Introduction and Philosophy • The introduction provides an overview of your organization’s PQI program or System. The introduction is a brief summary of the development and the history of the PQI program at the organization. If this is a new PQI program, the introduction can provide insight to how it was developed. This is an opportunity for the organization to discuss how the organization values and conducts activities for improvement. The introduction helps the reader understand the intent of the document and how it is helpful, whether the reader is a board member, community partner, client, employee, or volunteer. Strengths, accomplishments and challenges can be reviewed in this section. • The philosophy section provides an opportunity for the organization to define and communicate a PQI philosophy. There is an overview to how the organization’s leaders demonstrate a commitment to performance and quality improvement. The philosophy provides insight to how the Performance and Quality Improvement activities are integrated into the culture of the organization. The philosophy promotes excellence and continuous improvement. The philosophy is broad-based and inclusive of all stakeholders, programs, services, and locations. The philosophy advances effective services and has a positive impact on consumers. The philosophy emphasizes the use of data to build capacity, improve programs, and have a positive impact on persons served. II. Stakeholder Involvement • This section defines its stakeholders and how they will be involved in its PQI system. Stakeholders are the people who have an interest or “stake” in your organization’s success at achieving its mission. Stakeholders may include the Board, staff, consumers, funders, community, courts, etc. it is important Council on Accreditation www.COAnet.org 34 Version 1.02; Revised November 21, 2016 for organizations to understand that stakeholders’ involvement in PQI is fundamental to a well-designed program. A chart is a helpful way to illustrate to readers who the stakeholders are and how they interact with the organization. As previously discussed, the Stakeholder Charts can be inserted in this section of the PQI Plan. III. Structure • The organization describes the structure in this section of the PQI Plan. Regardless of the size of the organization, the structure has the capacity to receive data, identify opportunities for improvement and develop an action plan for change. The program is able to implement solutions to improve efficiency and deliver accessible, effective services to all regions and sites. For some organizations, the structure consists of a coordinator who is solely dedicated to the PQI program and for other organizations the PQI Coordinator may have responsibilities aside from PQI. Committees are often a critical component of PQI structure. This section of the PQI Plan documents how the organization builds an operational PQI structure that is sensitive to available resources and organizational need. A committee(s) can sometimes be built in to existing organizational committees or groups. Depending on the size and complexity of an organization, there may be more than one committee to address the multiple facets of PQI for the organization. IV. Model of Change • This section provides you with an opportunity to define the methodology or model that is employed to act on data from stakeholders. When trends, opportunities, or challenges are identified through analysis of data, the chosen method is used to guide the organization’s action to result in change. When change is implemented and the results are observed, it is important to note how it is determined if the change produced the intended results. Regardless of the outcome, the information is provided back to the stakeholders. This action closes the feedback loop and tells stakeholders that the organization is dedicated to continuous improvement. V. The Process • The process combines all of the individual components of the PQI Plan and demonstrates their relationship to each other. The flow of information should be evident and described to the reader. The chart on the following page shows all of the individual components as well as how information flows through the system. See infographic on next page or download here. Council on Accreditation www.COAnet.org 35 Version 1.02; Revised November 21, 2016 If an Improvement Plan is needed, the PQI Coordinator is typically involved and guides the process. The results are reported to the appropriate stakeholders. A culture of excellence and continual improvement supports the PQI infrastructure of the organization. The stakeholders provide the data and they receive the summarized information based upon the data that they provided. Council on Accreditation www.COAnet.org 36 Version 1.02; Revised November 21, 2016 VI. Measures • The organization should describe what is being measured in its PQI system. It is useful to clearly indicate funder and/or regulatory measurement requirements in the relevant sections. A comprehensive PQI Plan documents how each stakeholder in the PQI program provides valuable information. For organizations who have utilized the Program Indicators Worksheet to map out there measurements, this is a great place to insert them. • Remember, COA expects organizations to include the following measures in PQI Plans: 1. Impact of Services on Clients: There are numerous areas relating to the impact of services on clients that can be measured depending on the type of organization, capacity and size. Examples include: change in functional status; health, welfare, and safety; permanency of life situation; quality of life; achievement of individual service goals; and other outcomes as appropriate to the program or service population. This is also an opportunity to assess the level of satisfaction clients have with the service. 2. Quality of Service Delivery: Reviewing the quality of service delivery can focus on documentation, consumer satisfaction and timeliness. A quarterly file review measures the quality of service delivery by looking at compliance and consistency. Consumers can directly report their satisfaction to services by completing surveys or comment cards. Timeframes are reviewed to determine if consumers are receiving services in an appropriate amount of time after initial contact or wait list. Dimensions of quality can range and be tailored to the organization’s specific mission. At least on an annual basis, an organization needs to review the service delivery process. During this annual review, organizations can review data that is essential in their work and benchmark results. 3. Management/Operational Performance: Thorough review and benchmarking of administrative practices are equally important in the establishment of a PQI program and plan. Key administrative and operational practices that are included are human resource reviews such as retention and satisfaction of staff. Measurement of progress towards strategic goals is an opportunity to connect the Strategic Plan to the PQI Plan. Financial practices, cost analyses, review of financial audits and other facets of financial management provide a foundation for measuring performance of operations. 4. External Review Processes: External review processes make up the final component of measures and outcomes. While internal goals and objectives are important for an organization, ensuring compliance with external entities’ expectations is crucial to sustainability. Incorporating these expectations into a PQI Plan provides a well-rounded system of evaluation. Council on Accreditation www.COAnet.org 37 Version 1.02; Revised November 21, 2016