Program Evaluation

 Single-Group and Nonexperimental Methods for Outcome Evaluations: Interpretation of Findings

RFP#2

Topic: Prevention – Opioid Addiction

Program Evaluation Request for Proposal

Needs Assessment and Recommendations for Planning

Problem

The Merryville Public School System (MPSS) is soliciting proposals for a systematic evaluation of whether there is a need to offer preventive programs in their elementary grades to address opioid addiction.

Community health officials have reported increased opioid use and dependence among adults, including those who are parents of younger school-age children. Last year, the county health department began to provide informational posters and brochures for use in the district’s high school (grades 7–12, 160 students). Brochures also were made available to the parents of the high school students and through local businesses and religious organizations. However, it is unknown whether this method has been effective in general or among families with younger school-age children. To date, no direct discussions or education and prevention activities have been planned or developed for use within the district’s elementary school (K–6, 214 students).

Purpose of Evaluation

(1) Needs assessment: Before a prevention activity can be considered and planned, it is necessary to identify whether key stakeholders believe there is a need to offer activities to these younger students to address opioid addiction. Are these children actually confronted with opioid use/addiction within their homes? What are the views of school administrators, teachers, counselors, and other school personnel? Parents? Other key stakeholders in the community? What are their recommendations?

(2) Justification of cost: Is the need sufficient to justify the cost of developing such a program, training staff, and possible changes in school curriculum, policies, and other related aspects of implementation?

(3) Recommendations for planning: If there are identified needs, what models and goals do other school prevention programs like this address?1 Would they match this school’s needs, etc.? How are these kinds of programs implemented? What are steps to build into planning (e.g., who does what, training, staffing, other resources to implement)? What are the outcomes of similar programs and their impacts on communities? Are these feasible for this community?

Conditions of Contract

Budget for contractual evaluation and consultation set at maximum of $25,000. Report must be delivered no later than 6 months from the date of initiation.

Demographics Provided with the RFP

The MPSS serves a small rural community. There is only one high school and one elementary/middle school.

Student-teacher ratio across both schools is 14.9%. There are 25 classroom teachers (FTE). Currently, 31 students have active IEPs. For the elementary school, there are 9 classroom teachers (FTE), with a student-teacher ratio of 23.3%. For the high school: 16 classroom teachers, 10.0% student-teacher ratio).

The community demographics are as follows:

Gender: 57% male

Median age: 42.4 years (state median = 56.1 years)

Median household income (2017): $32,433 (down from $42,320 in 2014) (state median = $65,145)

Racial distribution: White = 94.0%, Hispanic = 3.1%, Black = 0.8%, Asian American = 0.5%, American Indian = 0.1%, two or more races = 0.6%, unknown = 0.8%

Educational attainment (those over 25): high school graduate or lower: 46.3%; some college/associate degree: 38.4%; bachelor’s degree: 12.2%; graduate degree: 3.1%

Unemployment rate: 12.4% (state = 4.2%) (rate up from 8.2% in 2014)

Religious affiliation (self-reported): 82% Christian, 2% Jewish, 4% other religious groups, 12% no religious affiliation

Recent estimates from the county health department:

Substance abuse rates (12 or older)

Alcohol use by youths aged 12–20 37.8%

Binge alcohol use by youths aged 12–17 5.5%

during previous month

Cigarette smoking 28.5%

Smokeless tobacco use 8.5%

Marijuana 11.2%

Illicit drug use 14.2%

Misuse of opioids 5.1%

Cocaine 1.1%

Crack 0.2%

Methamphetamine 0.9%

The county health department has more detailed data related to demographics of opioid use, treatment, etc.

1 Supplementary Resources

Operation Prevention. (2017). Classroom resources. https://www.operationprevention.com/classroom

Prevention First. (n.d.). Opioid education resources. Retrieved February 25, 2019, from
https://www.prevention.org/Professional-Resources/Opioid-Education-Resources/

U.S. Department of Education. (n.d.). Combating the opioid crisis: Schools, students, families. Retrieved February 25, 2019, from https://www.ed.gov/opioids

Sample Programs

Business Wire. (2018, September 13). D.A.R.E. launches new curricula for preventing opioid and prescription drug abuse.

https://www.businesswire.com/news/home/20180913005257/en/D.A.R.E.-Launches-New-Curricula-Preventing-Opioid-Prescription

Overdose Lifeline Inc. (n.d.). Home page. Retrieved February 25, 2019, from

https://www.overdose-lifeline.org/opioid-heroin-prevention-education-program.html

· Briefly restate the purpose of your team’s program evaluation RFP.

· Describe a situation for which a single-group design might be considered for an evaluation of the general program situation of your RFP (e.g., Opioid prevention). Why might it be appropriate in this situation? Explain.

· Given your example, explain the strengths and limitations of a single-group design for use in this evaluation.

· Would you choose a posttest only or pretest-posttest design? Why?

· How would you try to deal with threats to internal validity as part of your planning?

General Description

• Provide a general description of the program/project to be evaluated.

Need to Be Addressed

• Identify the specific problem to be addressed by the program activities and provide justification of that need.

Context

• Within what context is the program operating (i.e., setting; any environmental factors that may positively or negatively affect the initiative)?

Target Population

• Who is the target population of this program?

Stage of Program Development

• In which stage is this program at present: planning, implementation, or mature phase of program? Provide some details.