Community Support

Integrating Process and Outcome Evaluations

V. Developing a Computerized Database

Structuring an electronic database file and entering data complete the creation of the PDB. The range of database and statistical software currently available that would do an acceptable job of structuring the kinds of information in a PDB is very wide and changing rapidly. Because of this, we prefer not to make any specific recommendations about this software. Most database programs can produce files that are readable by other programs including statistical packages. The ability to generate reports, and convert files to statistical-package files for analysis should be a consideration, however. Data entry screens can easily be constructed that permit some checking of entered data for range errors and consistency with previously entered data, as in the following:

It is possible to have coders record their decisions directly into screens such as these, sharing accesses to the database files on a local-area network. An advantage of this approach is that it eliminates the step of transferring the data from hardcopy coding forms to a computer file. We have found, however, that coders find hardcopy forms useful in discussing coding difficulties and resolving disagreements that arise in reliability assessments. A mixture of approaches may be appropriate, when, for example, data from archival sources is combined with coding by analysts.

Reliability checking and other data quality control procedures are now standard features of meta-analytic coding (see Lipsey and Wilson, 1996; Orwin, 1994). We have typically conducted these inter-coder agreement studies at an early point in the coding process, and then continued spot-checking as the sometimes drawn-out project-level coding process proceeds. After establishing coding rules and discussion among coders, acceptable agreement, by one of the standard measures described in Orwin (1994) can be achieved in most cases. An assessment of the reliability of individual items can also be useful for identifying "questionable items" deciding to give these little weight in analyses of cross-site variation.

Table of Contents | Previous | Next