 |
This Web site is a component of the SAMHSA Health Information Network. |
|
Appendix B
PDF
version
You will need Adobe
Acrobat Reader to view this file.
Sources and Qualifications of the Data:
1997 Client/Patient Sample Survey
Survey Design
Scope of the Survey
The survey was conducted during 1997 and included all types of specialty mental
health care organizations located in the 50 States, the District of Columbia,
and the Territories. The types of organizations included in the survey were
State and county mental hospitals, private psychiatric hospitals, multiservice
mental health organizations, Department of Veterans Affairs medical centers,
non- Federal general hospitals with separate psychiatric services, residential
treatment centers for emotionally disturbed children, freestanding outpatient
mental health clinics, and freestanding partial care organizations. The survey
covered the inpatient, residential, and less than 24-hour care programs operated
by these types of organizations during a 1- month period in 1997.
The target population included two groups: (1) all persons newly admitted,
readmitted, or transferred into the program during a specified survey month
who were not already residents/on the rolls of the program on the first day
of the survey month, referred to as the admission population, and (2) all persons
who were admitted to the program before the first day of the specified survey
month and who received service from the program during the survey month, referred
to as the under care population. An oversample of children and youth under age
18 was included in the sample design so that reliable national estimates could
be generated for this specific population subgroup. Separate survey questionnaires
were designed to collect data from four groups-adult admissions, adults under
care, child admissions, and children under care, from within the inpatient,
residential, and less than 24-hour care programs of the mental health organizations
identified above.
The survey was conducted by the Survey and Analysis Branch (SAB), Division
of State and Community Systems Development (DSCSD), Center for Mental Health
Services (CMHS), Substance Abuse and Mental Health Services Administration (SAMHSA),
in cooperation with State mental health agencies.
Sampling Frame
The sampling frame for the survey was the 1994 Inventory of Mental Health
Organizations and General Hospital Mental Health Services (IMHO/ GHMHS).
Unique combinations of the eight organization types and three program types,
identified earlier, defined the 14 first-stage primary sampling strata (table
B1). The term "organization/program," used henceforth, refers to these combinations.
The measure of size used to stratify the programs was the number of persons
under care at the beginning of 1994 plus the number of admissions during 1994.
Sample Design: First-Stage Selection- Mental Health Organizations and Programs
The sample was based on a two-stage cluster design for all primary strata,
with the exception of primary strata 2, 9, and 10 (see Table
B1). For these strata, the sample design was a single-stage design with
all programs selected with certainty. Actual sampling was carried out in several
steps. First, to ensure geographic representation of the sample, programs were
arranged separately by region, by State within region, and by city within State.
A systematic sample of programs was then selected for each cell with a random
start in the first sampling interval. This sampling procedure was carried out
separately for organizations that operated one or two program types and those
that operated three program types (i.e., inpatient, residential, and less than
24-hour). This was done to reduce the burden on organizations so that no more
than two programs were selected from any given sampled organization.
For all primary strata, except Department of Veterans Affairs medical centers
(strata 9 and 10), which are exclusively for adults, most organizations/programs
treated both adults and children. A small number of organizations/programs either
treated adults only or treated children only.
The total number of programs sampled was 2,737, of which 228 were not within
the scope of the survey, that is, "out-of-scope" (e.g., program closed). The
overall survey response rate was 64 percent of the target sample. The final
column in table B1 presents
the number of organizations/programs that responded to the survey by primary
stratum.
Sample Design: Second-Stage Selection- Clients/Patients
For client/patient selection, separate listing booklets were used to establish
the sampling frame for each of the four groups (adult admission, adult under
care, child admission, and child under care) within each type of program (inpatient,
residential, and less than 24-hour). Using separate booklets for adults and
children under age 18, sample programs were asked to list the case numbers for
all persons newly admitted, readmitted, or transferred into the program during
the survey month who were not already resident/on the rolls of the program on
the first day of the survey month. Sample programs were also asked to list in
separate booklets for adults and children under age 18 the case numbers for
all persons who were admitted to the program before the first day of the survey
month and who received service from the program during the survey month. Programs
were asked to list case numbers only once in the booklets, and to include all
geographic locations of the program. Programs had the option of generating computerized
client/patient listings in place of manually completing the listing booklets.
Once the listings were completed, programs were asked to call a toll-free telephone
number to speak with a survey specialist. Using a specially designed computer
program to generate random numbers for the survey and using information obtained
directly from the program, the specialist selected "online" random numbers that
corresponded to completed line numbers in the program's listing booklets (or
computer-generated listings). The specialist informed the program as to which
line numbers were selected. The case numbers found on these line numbers identified
for the program which persons were to be sampled.
To reduce the burden on an organization/program, the total number of questionnaires
that were to be completed on persons sampled from all four groups was limited
to a predetermined number based on the size of the program. Smaller programs
were requested to complete a maximum of 8 questionnaires; larger programs a
maximum of 16 questionnaires.
Table B2 presents the number
of persons sampled and the number of respondents in each of the four groups
by primary stratum.
For strata 1, 3, 4, 6, 7, 8, 11, 12, and 14, children were oversampled at a
rate of three to one compared to adults. For strata 2, 5, and 13, children were
sampled at the same rate as adults. For strata 9 and 10, which refer to the
Department of Veterans Affairs medical centers, children were not sampled (i.e.,
not applicable).
Data Collection and Instruments
Data collection was accomplished primarily by mail, with telephone followup
to participating programs. Initial letters were mailed to the administrators
of sample organizations in March 1997 to inform them of the survey, its purpose,
anticipated levels of effort that would be required, and the program(s) in their
organization that had been selected for the survey. A followup call was made
to the administrators to discuss the survey further, answer questions, and request
participation. Numerous attempts were made by certified mail and telephone callbacks
to elicit survey participation. Prior to the survey month, a packet of survey
materials was sent to the designated person for each program that had agreed
to participate. The packets included all necessary survey forms (color-coded
listing booklets and corresponding questionnaires) and instructional material
(detailed instructions for completing the survey forms, procedures for selecting
the sample of persons, information on obtaining survey assistance, and instructions
on returning the completed survey forms in the postage-paid return envelopes
provided in each packet).
The data collection forms used for the survey focused on the sociodemographic,
clinical, and service use characteristics of persons. Inpatient and residential
sample programs used the same color-coded listing booklets and questionnaires.
These forms were similar in content to the forms used for less than 24-hour
care programs with slight variations in vocabulary to conform to different program
usage. Different colored forms were used to differentiate among the four groups:
adult admissions, adults under care, child admissions, and children under care,
and between inpatient/residential and less than 24-hour care programs.
Estimation
The sample for this survey was weighted to produce unbiased national estimates
about the number and characteristics of persons served in the inpatient, residential,
and less than 24-hour care programs of specialty mental health organizations
in the United States. Sample counts were inflated to national estimates in accord
with each stage of the sample design and nonresponse patterns. Hence, estimates
reported for admissions are weighted to 1- year totals; those for the under
care population to 1- day totals.
Limitations of the Design
Nonresponse
For this survey, nonresponse errors could exist in three ways: (1) failure
to obtain participation from some of the programs selected into the sample;
(2) failure to obtain data for some of the persons selected into the sample;
and (3) failure to obtain complete data for some sampled persons.
To minimize bias that might exist due to nonresponse, the information reported
by responding organizations was adjusted to compensate for program and person
nonresponse. The first-stage adjustment factor was the ratio of the number of
sampled programs (after removing the out-of-scope programs) to the number of
programs that responded. This adjustment factor was calculated and applied separately
to each stratum for each organization by program type combination. The second-stage
adjustment factor was the ratio of the number of sampled persons admitted or
persons under care to the number of corresponding person respondents, calculated
and applied separately for each of the four groups in each program respondent.
Missing items on the survey questionnaires were imputed using a sequential
hot deck procedure, as follows: Records were sorted on core sets of variables,
such as organization and program type, client/patient type, gender, age, diagnosis,
and region, to determine the imputation classes. The value of the variable from
the previous completed record in this ordered file was substituted for the unknown
value. After the sequential hot deck procedure was performed on a given variable,
a determination was made on how many times a given donor was used in the process.
If any donor was used five or more times during imputation of a particular variable,
a within-class random hot deck procedure was performed instead of a sequential
hot deck procedure to impute that variable. That is, records were sorted on
core sets of variables to determine the imputation classes. Then an observed
value of the variable was selected at random within that imputation class to
substitute for the unknown value.
Reliability of Estimates
Background
Because estimates presented in this report are based on sample data, they are
likely to differ from figures that would have been obtained from a complete
enumeration of the universe of specialty mental health organizations using the
same instruments. Results are subject to both sampling and nonsampling errors.
Nonsampling errors include biases due to inaccurate reporting, processing, and
measurement, as well as errors due to nonresponse and incomplete reporting.
These types of errors cannot be measured readily. However, to the extent feasible,
each error has been minimized through the procedures used for data collection,
editing, quality control, and nonresponse adjustment.
The sampling error (standard error) of a statistic is inversely proportional
to the square root of the number of observations in the sample. Thus, as the
sample size increases, the standard error decreases. The standard error measures
the variability that occurs by chance, because only a sample rather than the
entire universe is surveyed. The chances are about two out of three that an
estimate from the sample differs by less than one standard error from the value
that would be obtained from a complete enumeration. The chances are about 95
out of 100 that the difference is less than twice the standard error, and about
99 out of 100 that it is less than three times as large.
In this chapter, statistical inference is based on the construction of five-percent
confidence intervals for estimates (0.05 level of significance). All statements
of comparison in the text relating to differences such as "higher than" and
"less than" indicate that the differences are statistically significant at the
0.05 level or better. Terms such as "similar to" or "no difference" mean that
a statistical difference does not exist between the estimates being compared.
Lack of comment on the difference between any two estimates does not imply that
a test was completed and there was a finding of no significance.
Calculation of standard errors
Standard errors were calculated on a personal computer for a broad range of
totals and subtotals within age, gender, and race subclasses through the use
of SUDAAN Survey Data Analysis Software developed at the Research Triangle Institute
by B.V. Shah. This procedures computes estimated standard errors through the
use of Taylor series approximation. As applied to data from the present survey,
variance estimates for totals and subtotals were calculated for each stratum
and then summed across strata to derive standard errors for characteristics
of interest. The variance estimate for each stratum includes both the between-program
and the within- program components of variance, with corrections for finite
populations applied at both sampling stages.
Relative Standard Errors of Totals and Subtotal Estimates, Percentages,
and Rates
The relative standard error of a total or subtotal estimate, percentage, or
rate for a characteristic of interest is obtained by dividing the standard error
of the estimate by the estimate itself and is expressed as a percentage of the
estimate.
Relative Standard Errors of Differences Between Two Statistics
The standard error of a difference is approximately the square root of the
sum of the squares of each standard error considered separately. The relative
standard error of a difference is the standard error of a difference divided
by the difference.
Relative Standard Errors of Statistical Sums
The standard error of a sum of a number of independent estimates is the square
root of the sum of the squares of the standard errors of the separate estimates.
The relative standard error of the sum is the standard error divided by the
sum.
Table B3 presents standard
errors and percent relative standard errors for the estimated numbers, percentages,
and rates per 100,000 U.S. civilian population of selected major characteristics
for persons under care and admitted to inpatient, residential, and less than
24-hour care programs, for each type of organization surveyed. The statistics
presented in table B3 can be used to show the relative sizes of the characteristics
detailed in tables 1 through 19 of Chapter 15. The reader is cautioned that
if a relative standard error (i.e., the standard error of an estimate, percentage,
or rate divided by the estimate, percentage, or rate itself, expressed as a
percent) is 50 percent or higher, the estimate, percentage, or rate is not considered
reliable and should not be used.
Index | Previous
| Next
|
|
