SAMHSA's National Mental Health Information Center

This Web site is a component of the SAMHSA Health Information Network

  | | |      
Search
In This Section

Online Publications

Order Publications

National Library of Medicine

National Academies Press

Publications Homepage

Page Options
printer icon printer friendly page

e-mail icon e-mail this page

bookmark icon bookmark this page

shopping cart icon shopping cart

account icon  current or new account

This Web site is a component of the SAMHSA Health Information Network.


skip navigation

Section II: Supporting Good Decision-Making

PDF version
You will need Adobe Acrobat Reader to view this file.

Chapter 5. Twenty-five Years of the Mental Health Statistics Improvement Program: Past, Present, and Future- Where Have We Been and Where Are We Going?

Mary E. Smith, Ph.D.
Illinois Department of Human Services, Office of Mental Health

Steven Davis, Ph.D.
Oklahoma Department of Mental Health and Substance Abuse Services

Vijay Ganju, Ph.D.
National Association of State Mental Health Program Directors Research Institute

Leslie Tremaine, Ed.D.
Jefferson Parish Human Services Authority-Louisiana

Neal Adams, M.D.
California Department of Mental Health

Herbert (Chip) Felton, M.S.W.
New York Office of Mental Health

Olinda Gonzalez, Ph.D.
Substance Abuse and Mental Health Services Administration, Center for Mental Health Services

Judy Hall, Ph.D.
Washington Department of Social and Health Services, Mental Health Division

Cynthia Hopkins
Texas Department of Mental Health and Mental Retardation Services

Ted Lutterman
National Association of State Mental Health Program Directors Research Institute

Marla Smith, M.A.
Wyoming Department of Health, Mental Health Division

Boyd Tracy, M.Ed.
New Hampshire-Dartmouth Psychiatric Research Center

Ronald W. Manderscheid, Ph.D.
Center for Mental Health Services, Substance Abuse and Mental Health Services Administration

Introduction

What Is the MHSIP?

Over the past 25 years, the Mental Health Statistics Improvement Program acronym-MHSIP- has become increasingly recognized as an imprimatur of innovation and excellence in the area of mental health data systems and standards. The MHSIP has several facets: first, it is a policy group that provides at the national level leadership and direction related to mental health data; second, it is a community that comprises the policy group, regional user groups (RUGs) that represent all 50 States and U.S. territories, and other stakeholders with expertise and interest in mental health data and its uses; third, it is a set of values related to the effectiveness and efficiency of care that has evolved over time to be more consumer-centric and outcomes-oriented; and fourth, it is a set of products that have included data standards, reports on special topics, performance measurement systems, and report cards.

This chapter provides an overview of the evolution of the MHSIP, reviews past accomplishments, and describes future directions for the organization.

Historical Context

In 2001, the MHSIP celebrated its 25th anniversary. Cecil Wurster (1999), who for many years was the chief of the National Institute of Mental Health (NIMH) branch charged with the administration of the program, describes the development of the MHSIP as follows:

The MHSIP has its origins in the Model Reporting Area (MRA) for Mental Hospital Statistics, which was established by the National Institute of Mental Health and 11 state mental health agencies in 1951. The MRA was formed to address the need for uniform definitions among state mental hospital statistical systems. By the mid- 1960s, 34 states had met MRA statistical standards and had been admitted into the Model Reporting Area. However, with the expansion of outpatient and community- based mental health services in the late 1950s and early 1960s, the MRA was no longer sufficient for setting data standards for mental health services, because it continued to focus only on the state hospital systems. Consequently, in 1966 the MRA was essentially abandoned and the annual meetings of the MRA, supported by NIMH, were modified and expanded into a National Conference on Mental Health Statistics; and all states were invited to participate. During the next few years, the use of standardized definitions for mental health statistics decreased and the participants in the meetings of the National Conference appealed for the restoration of a mechanism to develop and promote standardization. At the 1976 meeting of the National Conference, NIMH proposed a cooperative mental health statistics program in the form of the Mental Health Statistics Improvement Program. The participating state representatives endorsed the concept of a cooperative system and agreed in principle to a collaborative relationship among the States and NIMH. In the collaborative relationship, the states agreed to work together with NIMH to develop standardized minimum data sets for mental health statistical systems and implement the data sets and data standards as statistical systems were developed or modified.

Initially, participants in the MHSIP were largely representatives of State mental health authorities (SMHAs) and NIMH. However, over the years, the MHSIP has expanded to reflect the diversity of persons who have an interest in and an expressed need for mental health data standards. These persons include consumers, family members of consumers, researchers, advocates, and mental health staff at the local, State, and Federal levels, encompassing both the public and private sector. The MHSIP has come to be known as a community of persons who are interested in "fostering and enhancing the quality and scope of information for decisions that will improve the quality of life and recovery of people with mental illness" (Wurster, 1999). One of the most important underpinnings of the MHSIP has been adherence to a set of values and beliefs that have driven the development of data standards and other products. These values and beliefs may be summarized in the following way: people need good, objective, reliable, and comparable information to make data-based decisions; good data depend on the use of data standards; and data are the basis and primary building blocks for accountability.

Besides the commitment to data, MHSIP values have evolved to emphasize the consumer perspective, outcomes, and broader stakeholder participation. This orientation is reflected not only in MHSIP products but also in the inclusive and consensus- building processes used to develop those products. Over the years, MHSIP values have been expanded to reflect (1) the increasing diversity of mental health stakeholders who use data for decisionmaking; (2) the increasing importance of performance measures and indicators as a basis for accountability and program improvement; (3) the rights of consumers and family members of consumers to be included in decisionmaking; (4) the recovery movement initiated by consumers; and (5) the recognition that access to high-quality data supports decisionmaking that can help improve the quality of life and recovery of people with mental illness. These enhancements have enriched and justified the primary value underlying MHSIP efforts, which is the promotion and advocacy of high-quality data to support decisionmaking.

Organizing Principles

The development of a quasi-formal organizational structure across the States has been fostered primarily through two mechanisms: (1) voluntary participation by State mental health authority staff in MHSIP-related activities, including the development of MHSIP-related products, and (2) States' receipt of federally funded grants that have promoted the enhancement of State data system infrastructure and the implementation of MHSIP products. Today, each State and U.S. territory has one or more identified MHSIP representatives. With the support of the Center for Mental Health Services (CMHS), these representatives convene in four MHSIP RUGs at least twice a year to share information, provide technical assistance to the States within the region, and promote the use of data standards. In addition, they express the needs and priorities of the field, which provide the basis for the policy decisions made and the direction established by the policy group. State MHSIP representatives are viewed as the implementers of many MHSIP products. That is, this cadre of people understands and applies MHSIP concepts to much of the work that has been undertaken in the field. These representatives are able to do this by advocating for the use of MHSIP products at both the State and local levels, training mental health system stakeholders with regard to the use of data, piloting data standards and performance measures, and modifying and developing information technology solutions that incorporate or are built on MHSIP values, data standards, and other products.

Over the years, MHSIP activities have been guided by a policy group that is supported by, and that works in collaboration with, CMHS. Members representing the broad spectrum of the MHSIP community described previously serve on the policy group. Since the inception of the MHSIP, this group (which is now known as the MHSIP Policy Group) has taken a leadership role at the national level, advocating for the development, adoption, implementation, and use of mental health data standards, as well as outcome and other performance measures that can be derived using standard data elements.

Past and Current MHSIP-Related Products

The MHSIP community has developed the following products:

The Design and Content of a National Mental Health Statistics System, FN-8 (1983). This document (Patton and Leginski, 1983) was the first MHSIP effort to identify, describe, and document data standards for three minimum data sets for mental health organizations: organizational, human resource, and client or patient data.

Data Standards for Mental Health Decision Support Systems, FN-10 (1989). This document (Leginski et al., 1989) updated the three data sets described in FN-8 and added two additional minimum data sets: event and financial. The FN-10 has been widely adopted by the States, and even today it continues to serve as the foundation for many SMHA information systems. It introduced the concept of collecting data to address a broad array of critical questions summarized as "who received what service from whom at what cost and with what outcome."

Enhancing MHSIP to Meet the Needs of Children (1992). This product dealt head-on with the expansion of MHSIP concepts and data standards to address mental health services provided to children and adolescents and their families. Three problems were addressed by the Task Force on Enhancing MHSIP to Meet the Needs of Children (Cole et al., 1992): (1) MHSIP content and data standards were focused primarily on adults; (2) the MHSIP definition of the mental health service delivery system focused on the use of the specialty mental health service system as the vehicle for service delivery, but the task force noted that mental health services are provided to children and adolescents and their families across service systems-not just within specialty mental health; (3) the provision of services to children across systems results in unique issues with regard to data integration efforts. Task force recommendations were formulated to address each of these issues and next steps were identified for implementation.

Performance Indicators for Mental Health Services: Values, Accountability, Evaluation, and Decision Support (1993). The focus of this product was to develop a conceptual framework and to recommend a process for the development of performance measures and performance indicator systems based on MHSIP data sets and data standards content that could be used to "support accountability, evaluation and data-based decision-making" (Task Force on Performance Indicators for Mental Health Services, 1993).

Commissioned Reports

Several times over the past 25 years, with support from SAMHSA/CMHS and other sources, the MHSIP Policy Group has solicited preparation of papers to summarize the salient sources of influence for a particular issue or problem. These commissioned reports were often very influential in guiding the decisions and direction of the Policy Group. Three are described here, though several others have been produced (and are available in the Library pages of the MHSIP Web site, described below).

In 1993, two papers focused on trends and points of view that were to channel MHSIP resources and shape several MHSIP endeavors. One described the movement in health and other disciplines toward a more person-centered system of care, in which there is "recognition of the legitimacy and value of having persons actively participate in making decisions related to their well-being" (Buckley, 1993). This report concluded by recommending revising MHSIP data standards to reflect accountability to a broader range of stakeholders' decision support needs; adding a focus on consumer preferences and satisfaction; emphasizing consumer outcomes over time, rather than only the proximal outcome of an intervention; increasing the range of organizations included; promoting use of technology to enable appropriate linkage of person-specific data across time and settings, with sufficient provisions for privacy, confidentiality, and security of the data; and providing means and opportunities for access and effective use of data by consumers, family members, advocates, and other nontraditional stakeholders of the information system.

The second paper (Campbell and Frey, 1993) took another step along that evolutionary path to recommend a "person-driven decision support system" that ensures consumer dignity, respect, acceptance, integration, and choice in professional practice; avoids psychiatric labeling; educates about the utility of data; emphasizes health-promoting lifestyles and help-seeking behavior; encourages all stakeholders to serve the self-defined needs of consumers; provides protection of personal data; ensures access to data; promotes collaboration among policymakers, service providers, and service recipients; and provides funding to support data system change to a person-driven model.

Although they approached the subject from different perspectives, these papers contributed to important changes in the MHSIP philosophy. Because of them, more consumer membership was added to the Policy Group and subsequent initiatives have all been structured to ensure more consumer voice.

In the early to mid-1990s, support for the concept of recovery from mental illness began to build momentum. A third influential report for MHSIP- funded by CMHS, the National Technical Assistance Center for State Mental Health Planning (NTAC), and the Human Services Research Institute (HSRI)-summarized newly developing work, published and unpublished, on conceptualizing and measuring recovery among persons with mental illness (Ralph, 2000). The recommendations from this document (inclusion of consumers/survivors in the research process, collaborative development of measurement tools, documentation of attempts to establish recovery environments, and continued compilation of recovery literature) provided direction for subsequent recovery research to be included in the next MHSIP Report Card (see the discussion below).

The MHSIP Consumer-Oriented Mental Health Report Card, 1996

The MHSIP Consumer-Oriented Mental Health Report Card was designed as a prototype report card that could be used by consumers, mental health advocates, health plan purchasers, providers, and State mental health authorities to assess the quality and cost of mental health and substance abuse services (Task Force on a Mental Health Consumer-Oriented Report Card, 1996). The report card was unique in that it was developed on the basis of consumer concerns, and it focused on outcomes. As noted in the final report of the Task Force, the

focus on consumer needs goes beyond the inclusion of consumers in developing and evaluating the report card's indicators and measures. Indeed, the domains, concerns, indicators, and measures of the MHSIP report card are specifically designed to assess consumer concerns with various aspects of mental health treatment, not merely global satisfaction with mental health services.

The indicators and measures developed were designed to be "consumer-oriented, based on research and explicit values, focused on, but not limited to, serious mental illness, designed to emphasize the outcomes of mental health treatment, and conscious of related costs and staff burden."

The conceptual framework for the report card revolved around four domains associated with consumer concerns: (1) access to treatment, (2) appropriateness of treatment, (3) outcome of treatment, and (4) prevention. The report card relies on data collected from administrative databases, consumer surveys, and clinician-administered instruments.

Today, more than 45 States and territories have implemented various aspects of the MHSIP Consumer-Oriented Report Card. Many of these efforts have been supported partially or totally by funding from the Substance Abuse and Mental Health Services Administration's (SAMHSA's) CMHS. The consumer survey alone has been implemented by 44 States; it is also the basis for several indicators being reported under CMHS's funded Data Infrastructure Grants that have been awarded to 50 States and seven U.S. territories. Additionally, many measures and surveys, as well as a variety of initiatives adopted across the public and private sector, are derivatives of the content of the report card or other work undertaken by the MHSIP community.

The Enrollment-Encounter Task Force and FN-11

During the expansion of managed care that occurred in the 1990s, the MHSIP Policy Group established an Enrollment-Encounter Task Force to review data elements and organizational structures recommended in FN-10 to determine changes needed to ensure that the MHSIP data standards matched the needs of mental health providers and related entities operating in the managed care environment. The Task Force brought together representatives from managed care organizations (MCOs), the Health Care Finance Administration (now the Center for Medicare and Medicaid Services [CMS]), and other State, Federal, public, and private organizations affected by reporting needs under managed care. This effort to include private providers and MCOs was a distinct departure from the past, but was part of the evolution of MHSIP to a more inclusive, data-standard organization. The task force developed a survey, and, with the help of the American Managed Behavioral Health Association (AMBHA), collected data from a number of managed behavioral health organizations to identify data elements they found necessary to run their organizations. The output of the task force made it clear that a significant revision of FN-10 data standards was needed. As a result, the FN-11 Workgroup was established by the MHSIP Policy Group in 1996. The workgroup's charge was to revise and update the data standards that were published in the original FN-10 document described above. This task was to be accomplished by reviewing and incorporating the data system recommendations in FN- 10 and those in other MHSIP documents that had been produced since 1989 to address the needs of children's service systems, performance indicator systems, report cards, managed care, and work being done at the same time by the National Committee on Vital and Health Statistics. A revised set of data tables was compiled and reviewed with stakeholders at regional and national meetings, with the intent of publishing the results. However, the rising awareness of the impact of the Health Insurance Portability and Accountability Act (HIPAA), the initiation of the Decision Support 2000+ project, and the growing recognition of the World Wide Web as an effective tool for disseminating rapidly changing information led to output from the task force and the FN-11 Workgroup being folded into subsequent MHSIP initiatives, which are described below.

MHSIP Web Site (1998)

The MHSIP Web site, which is supported partially by the SAMHSA CMHS and partially by the Oklahoma Department of Mental Health and Substance Abuse Services, was initially developed as a mechanism for sharing information and for serving as a repository for work products developed by the FN-11 Task Force. Since 1998, the MHSIP Web site (www.mhsip.org), also known as MHSIP Online, has developed into a major vehicle that includes information on MHSIP-related products (nearly all products described in this chapter are available online) and descriptions of past and current projects; links to all MHSIP regional user groups and MHSIP Policy Group members; links to important issues that have an impact on data standards, such as HIPAA, as well as to related sites; CMHS data-related initiatives, such as Decision Support 2000+ (DS2000+); a calendar of MHSIP and other data-related meetings; and a vast library of documents that are available free to anyone with Internet access. MHSIP Online receives between 40,000 to 50,000 hits per month, which is a testament to its perceived usefulness among people who have an interest in mental health data standards.

Initiatives Based on MHSIP Products

One major impact of MHSIP activities has been made through the guidance MHSIP products have offered to Federal and State initiatives related to mental health data and information systems. Two sets of 3-year grants to State mental health agencies promoted the incorporation and use of MHSIP data standards in data systems and related initiatives; a 3-year State reform grant initiative promoted the implementation of performance measures proposed in the MHSIP Consumer-Oriented Report Card; and, most recently, grants from CMHS related to the Five-State Feasibility Study, the 16-State Performance Indicators Study, and the recent round of State Data Infrastructure Grants to 50 States and seven territories have many of their performance measures and reporting requirements based on MHSIP products and recommendations.

MHSIP activities have also influenced activities beyond the public mental health sector. For example, several mental health performance measurement activities, including those by the American College of Mental Health Administration (ACHMA), AMBHA, and the National Committee for Quality Assurance (NCQA) have based their work to varying degrees on MHSIP concepts, products, and recommendations. Other initiatives that have been based on MHSIP concepts and products include the following:

  • The Evaluation Center at Human Services Research Institute Toolkit on Performance Measurement Using the MHSIP Consumer- Oriented Report Card: a practical document describing "How to Implement" various aspects of the report card. HSRI collaborated with a number of persons in the MHSIP community to develop the toolkit.
  • The National Association of State Mental Health Program Directors (NASMHPD) President's Task Force on Performance Indicators expanded the framework of the MHSIP Report Card and identified additional indicators of importance for the management of State mental health systems.
  • Decision Support 2000+, an ambitious undertaking of CMHS, is an integrated set of mental health data standards and an information infrastructure designed to help all stakeholders answer key questions and make critical decisions that will improve the quality of care. DS2000+ consists of several components: population, enrollment, encounter, financial, organizational guidelines, and human resources data sets; performance indicators, report cards, and outcome measures. DS2000+ grew directly from the FN-10 data standards and the recommendations of the FN-11 Workgroup, as well as from HIPAA data requirements. Members of the MHSIP Policy Group and MHSIP RUGs have been involved in the design, development, and implementation of DS2000+ since its inception.

Future Directions

The MHSIP has endured 25 years because the diverse groups that make up the MHSIP community, as well as others with an interest in data standards and mental health, have recognized its usefulness. However, the MHSIP must adapt to, and reflect the changes in, the environment to remain relevant. In the fall of 2001, the MHSIP Policy Group convened a workgroup to begin planning the revision of one of its most well-known and embraced products, the MHSIP Consumer-Oriented Report Card.

Revision of the MHSIP Consumer-oriented Report Card

The MHSIP objective is always to provide useful, reliable information for consumers, their families, mental health service oversight authorities, providers, and other stakeholders who want to promote evidence-based recovery, quality care, accountability, and system improvement. The MHSIP Report Card, Version 2.0, will contribute to attaining that objective. The purpose of the revision is to consolidate the lessons and experiences of those who have developed and implemented mental health performance measures and to compile a set of measures that reflect key concerns of stakeholders who want to improve the performance of mental health systems or organizations.

Values Underlying Report Card, Version 2. The MHSIP Report Card, Version 2.0, is value-based. A central tenet of the MHSIP Report Card, Version 2.0, project is that a mental health performance measurement system should be designed and built to be consumer-centered and help consumers move toward recovery.

Guiding Principles for the Revision. The approach to the revision is conceptually different from the previous work undertaken to produce Version 1 of the report card. When the initial work was begun on the report card in 1993, the development of report cards for the health care sector was only just beginning. There was little experience to build on because the efforts under way were only in their initial developmental or early implementation stage. The passage of time has given us the opportunity to add substantially to our knowledge regarding report cards and performance measurement systems. As a result, the revision can be built on lessons learned by entities that have implemented report cards and performance measurement systems over the past nine years. MHSIP Report Card, Version 2, will also be a consolidation effort.

Collaboration and Partnering with Expert Groups. The revision effort will incorporate the work of other expert groups that have developed performance measures relevant to the goals and purpose of the report card. We have no need to reinvent the work of these groups, but rather to partner with them and collaborate to develop a useful product. One such group, whose work is of particular interest, is the Forum on Common Performance Measures in Behavioral Health and Related Systems. The Forum is an ambitious undertaking to "identify, develop, and implement [a small set of] common performance indicators and measurement specifications that are applicable to both public and private organizations and delivery systems that will be used across the entire mental health field" (Adult Mental Health Workgroup, 2002). The work of the Forum will be incorporated into Version 2 of the Report Card.

Modularity. Version 1 included a fairly large number of performance measures and indicators, yet lacked clarity regarding the population to which some indicators applied. Nor was it always clear to which settings the measures and indicators applied. As a result, some measures have been used with populations or settings for which they were not intended. The concept of modularity will be a defining factor in the revision and is based on the recognition that different sets of measures may be needed for different populations in different settings. However, a major emphasis of this work will be to develop consistency and commonality across these settings and populations.

The Report Card as a Framework for Assessing Quality and Cost of Care. The philosophy underlying the development of the original Consumer-Oriented Report Card will continue to be a driving factor in the development of Version 2. That is, "it uniquely reflects the needs of mental health consumers," and "it can be used by a broad constituency to determine whether a mental health system is meeting the needs of adults and children with mental disorders" (Task Force on a Mental Health Consumer-Oriented Report Card, 1996). If the goal is to use a report card for this purpose, a minimum number of performance measures and indicators across domains must be assessed. To use less than this minimum provides a disjointed and incomplete view of the service delivery system that is being assessed. The final version of Report Card, Version 2, will contain recommendations related to this philosophy.

Measures and Indicators for Children, Adolescents, and Their Families. A concerted effort will be made to include measures in the revision that are important to children and adolescents and their families. To ensure that this happens, members of the Outcomes Roundtable for Children and Families and the Federation of Families for Children's Mental Health have been recruited to be members of the MHSIP Report Card, Version 2, Workgroup.

Recovery. As of 1996, work had not been completed to operationalize the measurement of recovery, although the task force knew this was an important issue that needed to be addressed. The revision will incorporate work based on the findings and research being performed by the Recovery Facilitating System Performance Indicator Research Team. This research effort grew out of the 16-State Performance Indicator Study and has thus far focused on "what helps and what hinders recovery." The two co-principal investigators on the project are members of the Report Card Workgroup.

Consumer Surveys. The MHSIP Consumer Survey has been widely implemented across the United States. In addition to its use in public settings, it has been adopted for use in private sector settings. The survey was the basis for some of the developmental work on the Experience of Care and Health Outcomes Survey (ECHO) initiated by Harvard University with support from NCQA, with participation of members of the MHSIP Policy Group. The second-generation consumer survey will build on all this work, as well as reflect the collaboration now occurring between the mental health and substance abuse fields, incorporating interests of both the public and private sector.

Work on surveys designed for use with adolescents and family members of children and adolescents has also been initiated over the past three years. This work was undertaken as part of the 16- State Indicator Project, resulting in Version 1 of the Child/Adolescent MHSIP Survey, which is alternately known as the Youth Services Survey (YSS) and the Youth Services Survey for Families (YSS/F). Surveys incorporated in Report Card, Version 2, for youth and their caregivers will also reflect the collaboration between the mental health and substance abuse fields, cutting across the public and private sector.

Toolkit to Guide Implementation. When the first report card was published in 1996, it included no explicit guidance regarding how the various measures and indicators should be implemented or what constituted the best methodologies for data collection. A toolkit was developed by HSRI in collaboration with the MHSIP community within two years of the release of the report card. However, by this time, much testing of the report card measures had already been completed. The impact of the absence of guidance regarding the implementation of measures was that many measures were implemented differently and, in some instances, changed to reflect local needs and concerns. The overarching result is that it has been difficult to completely assess the impact of the implementation of Report Card, Version 1, across all States that implemented all or various components of it. The revision will address this issue by developing a toolkit in a process that occurs in parallel with the development of the revised report card. HSRI is supporting this aspect of the project by working in collaboration with task force members and others who are part of the MHSIP community. The Revised Toolkit will rely on use of innovative technologies to disseminate information that includes online access to material, interactive technical assistance functions, and links to other sites to enable access to other valuable resources that have been developed by others in the field.

Incorporate New Technologies for the Implementation of Performance Measures and for the Dissemination and Distribution of Reports. The revision will also address the use of innovative technologies to collect performance measurement data and to disseminate reports in user-friendly formats. Increasingly limited human and financial resources make it imperative that we take advantage of 21st-century technology to collect data and share information.

MHSIP Report Card, Version 2, Task Force

The MHSIP Report Card, Version 2, Task Force was constituted in November 2001. In line with its stated goals to be relevant to, and meet the needs of, the mental health field, representatives from the public and private sectors cutting across the diversity of mental health stakeholder groups were recruited (see list in table 1). The task force has met five times during the past year. The status of the revision of the report card is described below.

Report Card Status as of November 2002. A combination of several methods was used to consider and select indicators for inclusion in Version 2 of the report card. First, members of the workgroup were asked to describe "lessons learned" from the performance measurement initiatives in which their constituents had been engaged. They also were asked to identify performance measures they thought would be useful to consider for inclusion in Version 2 on the basis of their experiences with these initiatives. Several workgroup members representing groups that were actively working to develop performance were asked to discuss how their initiative could relate to the goals of the report card workgroup and to share information at the point that interim products were developed.

The second method consisted of the workgroup systematically reviewing a group of performance measures and indicators that are used across performance measurement systems. For this purpose, a matrix was constructed listing the performance measures currently used by each system or initiative. It was then possible to identify which measures had been adopted by multiple performance measurement systems. On the basis of this information, the workgroup selected a number of measures and indicators to be considered for inclusion in Version 2.

The third method built on the previous two methods. Each individual in the workgroup was asked to identify additional measures that should be considered for possible inclusion in Version 2 that had not been identified using the previous two methods. This brainstorming process resulted in the generation of additional measures and indicators (approximately 46), some of which were subsets of measures and indicators previously considered.

The workgroup systematically reviewed the concerns and rationale of each measure and indicator from Version 1 of the MHSIP Report Card. The purpose of this review was to determine whether the concerns were still relevant, or had adequately been addressed over time, and to discuss alternative ways to address the concerns if the original report card measures and indicators were not acceptable.

Finally, all this material was reviewed by the workgroup and consensus was reached regarding the proposed performance measures. At this stage, three types of indicators have been proposed: indicators that are currently in use that have standardized definitions; indicators that are being used but have multiple definitions; and indicators that require measures to be developed. The populations and settings to which they apply have been identified for each set.

Next Steps. Activities up to this point have involved the consolidation of information from various sources regarding the feasibility, utility, and need for indicators and measures by the Report Card Workgroup. As the next stages of the project unfold, broader input from consumers, family members, and other key stakeholders is being actively sought. An intensive effort will be mounted across the varied interests in the spectrum of stakeholders to actively seek feedback on each set of indicators. The goal of this activity is to obtain consensus regarding which ones are critical, in terms of importance and usefulness, to include in Version 2 of the report card.

Finalization of Proposed Measures and Indicators. Once this broad-based feedback is obtained, the task force will use the information to finalize the selection of measures and indicators that make up Version 2. It is expected that some of the measures will be those that have already been tested widely in the field. Others will need to undergo testing to determine how they may best be implemented.

Timelines. An initial working version of the revised report card is expected late in 2004. This document will be a working version in the sense that testing may not be completed on some measures for which it is warranted, and the work involving partnering with other expert groups may need to follow the other groups' schedules. For these reasons, some measures may still be in a developmental phase, and placeholders likely will need to be used until work on the measures is finalized.

Other MHSIP Projects

Continued Collaboration on Decision Support 2000+. As noted previously, members of the MHSIP Policy Group, members of the MHSIP RUGs, and other members of the MHSIP community have been involved in the design and development process for DS2000+. A formal mechanism, the MHSIP/Decision Support 2000+ Liaison Group, was established by the MHSIP Policy Group in February 2001 for this purpose. Members of the liaison group are being joined by others in the MHSIP community to work on specific components of DS2000+, including the information technology components. Also, work completed under the MHSIP Report Card, Version 2, project (including the second-generation consumer surveys that are being developed) is expected to be adopted as a prototype report card for DS2000+.

Behavioral Health Care Data Standards. There is a need now more than ever to ensure that the mental health field is anchored by a set of data standards that are rational, are well-defined, and reflect the values on which the field is based. The implementation of HIPAA regulations that aim to simplify the electronic transmission of health insurance information has as its foundation a set of data standards. A major issue for the behavioral health field has been its lack of representation in the process of setting the behavioral health care data standards for which it will be accountable. Over the past 2 years, members of the MHSIP community have collaborated with partners in the field, such as SAMHSA, NASMHPD, and the National Association of State Alcohol and Drug Abuse Directors (NASADAD) to ameliorate this problem and will continue to do so in the future.

Challenges for MHSIP

Although MHSIP has had success in meeting its goals to advance the use of mental health data standards and information technology to support rational decisionmaking, the challenges for MHSIP in today's complex world are many.

MHSIP Influence on Federal Data Initiatives

In the past, the MHSIP Policy Group has had a more prominent place at the table in discussions of guidance for Federally funded data initiatives. To some extent, the MHSIP has been negatively affected by the success of its own philosophy. The inclusion of diverse stakeholders in mental health data projects and standard setting processes funded by the Federal government has diluted the direct impact the MHSIP has on decisionmaking about these activities. Nevertheless, the MHSIP community continues its support of MHSIP values, and its history, strengths, and skills continue to offer a valuable resource and perspective for Federal initiatives.

Organizational Structure

The informal organizational structure of the MHSIP community and the MHSIP Policy Group described previously in this chapter presents a challenge. Although this structure has been functional and productive in the past, today's environment requires a more formal structure-one that lends itself to seeking and acquiring funding from a variety of sources to support MHSIP projects. Additionally, in the past, many MHSIP products were developed by volunteers. The economic climate has had a major impact on staffing in both the public and private sectors. As a result, it is more difficult for employees who have received additional responsibilities to "carve out" time to participate in voluntary activities, such as product development. The MHSIP, at this point, requires investment in dedicated staffing to support its efforts.

Public/Private Mental Health/Behavioral Health/Primary Health Partnerships

As stated earlier, the MHSIP's early activities focused on public sector mental health initiatives. Over the years, though, the MHSIP has increasingly become focused on data initiatives that span the public and private sector because no clear demarcation exists for many mental health consumers between the two service delivery systems. In other words, the funding source may be different, but both public and private sector providers now offer needed services. If one is consumer-focused and the notion of data supporting decisionmaking is endorsed, then it does not matter if the data are captured by public or private sector providers-what matters is that data from all viable sources must be used. Clearly, using data from one set of providers when multiple sets of providers exist distorts the view of reality.

Closely related is the need to focus on behavioral health as opposed to mental health. Many mental health consumers have co-occurring problems that require them to access other behavioral health services in addition to mental health services. The relationship of behavioral health to primary health services must also be considered as we look at data use for decisionmaking. Data can no longer be compartmentalized by single service delivery systems when people are accessing multiple systems for services. With the complexity of all these factors as a backdrop, the MHSIP faces the challenge of working within a broader context with more players across a spectrum of services. It is essential that the MHSIP goal of inclusion be a driver with regard to its future work.

Access to Funding

The MHSIP, for all of the volunteer work that has occurred over the years, is not immune to the need to access funding to support its work. The MHSIP needs to identify a range of resources, consisting of public and private sources, to maintain its ability to advocate for data and data standards at the national, State, and local levels and to support work performed by experts in the field.

Recognition of MHSIP Expertise

Despite the fact that the MHSIP has been at the forefront of data initiatives for 25 years and that it has the stature and expertise to continue this role, it is becoming increasingly difficult to gain recognition for this fact and to be invited to the table at which broader data initiatives are discussed. The MHSIP needs more visibility, greater support, and increased recognition of its expertise and skills to address accountability issues that are now being raised at the national level. It is a waste of resources not to use the strength and talents of the MHSIP community that have been developed and honed over a 25-year period.

The MHSIP Vision for the Future

Addressing the described challenges quickly and effectively will position the MHSIP to play a crucial role in guiding major policy work now underway in publicly funded mental health services. The high-level policy attention being paid to these issues in the wake of the recent Surgeon General's report and President Bush's New Freedom Commission on Mental Health Commission creates both opportunities and pressing needs for the guidance and support of a group with the history and expertise of the MHSIP. The MHSIP's vision as it relates to the current national agenda is essentially a dual one: that evolving mental health policies and practices be based on the best possible evidence and data base, and that mental health systems demonstrate full accountability to a range of stakeholders. Through the development of a strong learning community over the past 25 years, MHSIP has made major contributions to achieving this vision. Meeting the current challenges successfully will further the MHSIP's role in the 21st century in creating the data infrastructure for a strong mental health service system.

REFERENCES

Adult Mental Health Workgroup (2002). Background and goal statement. www.mhindicators.org.

Buckley, Susan M. (1993). Moving MHSIP toward a person-centered paradigm. Unpublished manuscript.

Campbell, J., & Frey, E. D. (1993). Humanizing decision support systems. Unpublished commissioned manuscript.

Cole, R. F., Bullard, A., Carroccio, D. F., Crosby, M., Evans, M., Hoagwood, K., et al. (1992). Enhancing MHSIP to meet the needs of children. Unpublished manuscript.

Leginski, W. A., Croze, C., Driggers, J., Dumpman, S., Geertsen, D., Kamis-Gould, E., et al. (1989). Series FN- 10, data standards for mental health decision support (DHHS Publication No. ADM 89-1589). Washington, DC: Supt. of Docs., U.S. Government Printing Office.

Patton, R. E., & Leginski, W. A. (1983). Series FN-8, the design and content of a national mental health statistics system (DHHS Publication No. ADM 83-1296). Washington, DC: Supt. of Docs., U.S. Government Printing Office.

Ralph, R. O. (2000). Review of recovery literature: A synthesis of a sample of recovery literature 2000. Alexandria, VA: National Technical Assistance Center, National Association of State Mental Health Program Directors.

Task Force on a Mental Health Consumer-Oriented Report Card. (1996, April). The MHSIP consumer-oriented mental health report card. Rockville, MD: Center for Mental Health Services.

Task Force on Performance Indicators for Mental Health Services. (1993). Performance indicators for mental health services: Values, accountability, evaluation and decision support. Unpublished manuscript.

Wurster, C. R. (1999). The mental health statistics improvement program: Background information for MHSIP focus groups. Unpublished manuscript.

Index | Previous | Next

Home  |  Contact Us  |  About Us  |  Awards  |  Accessibility  |  Privacy and Disclaimer Statement  |  Site Map
Go to Main Navigation United States Department of Health and Human Services Substance Abuse and Mental Health Services Administration SAMHSA's HHS logo National Mental Health Information Center - Center for Mental Health Services