Program Evaluation in Pain Medicine

There has been a rapid change in the way health care services are offered in the United States. More and more decisions about treatment are made by employees of insurance carriers on the basis of financial resources rather than need. "Managed care" favors structured, group-based programs because they are time-limited and economical to run. However, the increasing need for accountability and efficacy has encouraged the implementation of cost-saving measures and ongoing evaluation. Preference is given to programs that are of demonstrated efficacy and are tailored to the individual (e.g., not all participants receive every treatment).

Program evaluation improves the use of resources, organizes data for marketing, and helps managers decide which direction a program will take. Effective program evaluation reports offer data to assist top level administrators to create solutions for poor outcome, increased costs, and decreased market share. The goals of program evaluation in chronic pain management programs are to increase the benefits to persons the program serves, increase program productivity, and improve market position.

An important component of any group-based pain program is its ability to measure its own effectiveness. A number of recommendations for effective program evaluation have been put forward by the Commission on the Accreditation of Rehabilitation Facilities (CARF). Each program should have a mission statement, and lists detailing admission criteria, services offered, specific objectives, the priority of each objective, expectations regarding minimal and optimal goals, objective measures of performance, the period to which these measures apply, and the way in which information collected is documented.

A system should be in place for obtaining follow-up information from patients on the use of medications, the use of health care services, return to gainful employment, functional activities, the ability to manage pain, and the subjective intensity of pain. This system should include a schedule for periodic contact with the patient after discharge, and a data base containing information updated on a regular basis. This type of system helps determine how a program meets the needs of individual patients and of participants overall.

Program evaluation should encompass goals and objectives that are achievable and end results that are measurable. Each program evaluation system should include objectives and measures for 1) productivity of patients (e.g., return to work), 2) heath care utilization (e.g., reduction in physician visits), 3) activity level (e.g., increased walking and exercising), 4) medication usage (e.g., adherence to AMA guidelines for proper use of medications), 5) program costs (e.g., maintenance of estimated and actual costs per patient at a steady level), 6) program productivity (e.g., a low ratio between the number of staff hours devoted to the program and the number of patient hours), 7) patient helpfulness ratings (e.g., above average ratings of helpfulness for all services rendered), 8) pain coping and emotional adjustment (e.g., decreased ratings on the Beck Depression Inventory before and after treatment), 9) medical findings (e.g., improvement in objective physical measures, such as range of motion), and 10) socialization and activities of daily living (e.g., improved scores on the Sickness Impact Profile). A program evaluation report should include primary objectives, measures, time of measurement, source of information, and expectations as well as outcomes. Finally, program evaluation should help identify which services are most effective in the treatment of chronic pain patients.

Additional components of a program evaluation system include 1) follow-up data from a representative sample of patients, 2) valid and reliable dependent measures standardized for persons with pain, 3) a comparison of post-treatment and follow-up data, and 4) assessment of individual differences.

This article was excerpted from the book Mastering Chronic Pain: A Professional's Guide to Behavioral Treatment. by BW Pain Management Center physician Dr. Robert N. Jamison, Ph.D., published by Professional Resource Press, Sarasota, FL, 1996.