Agents of Change ASWB exam prep has helped 100,000+ social workers pass, with video and audio lessons, hundreds of practice questions, and live study groups. Start with 10 free practice questions.
Program evaluation and research questions show up on every level of the ASWB exam, and they make many test takers nervous. The good news: the exam doesn’t ask you to run statistics. It asks you to make practice decisions. Which design fits this question? Is this measure trustworthy? Is the program actually working? Did participants give real consent?
This guide covers the six steps of program evaluation, the types of evaluation, and the research basics most likely to appear on the new ASWB exam, with practice questions at the end.
1) What Is Program Evaluation?
Program evaluation is collecting information about a program in order to make decisions about it. Social workers evaluate programs to answer questions like:
- Was the program effective? Did it achieve its goals?
- Is it being delivered the way it was designed?
- Who is it reaching, and who is it missing?
- How much does it cost, and are the results worth it?
- Should the program be continued, changed or ended?
Types of Program Evaluation
Different evaluation questions call for different types of evaluation. On the exam, match the type to what the agency wants to know and when.
| Type | The question it answers | When |
|---|---|---|
| Needs assessment | What problem exists, and who needs services? | Before a program starts |
| Process (formative) evaluation | Is the program being delivered as planned? How can we improve it? | While the program runs |
| Outcome (summative) evaluation | Did the program achieve its goals? | At the end of a program or cycle |
| Cost-effectiveness / cost-benefit analysis | Are the results worth what they cost? | Usually after outcomes are known |
Outputs vs. Outcomes
Outputs are activities: sessions held, brochures handed out, people enrolled. Outcomes are results: people housed, symptoms reduced, school attendance improved. Outputs show a program is busy. Outcomes show it’s working. When an exam question asks whether a program achieved its goal, the right measure is almost always an outcome.
2) The 6 Steps of Program Evaluation
A widely used framework for program evaluation, developed by the CDC, breaks the process into six steps. Here’s what each one means, using a sample program: an after-school mentoring program designed to improve attendance for middle school students.
Step 1: Engage Stakeholders
Bring in the people who care about the program and its results: participants, families, staff, funders and community partners. Their input shapes which questions matter and helps make sure the findings get used.
Example: The social worker meets with students, parents, teachers, mentors and the funding foundation to learn what each group hopes the program will achieve.
Step 2: Describe the Program
Clearly explain what the program is: the need it addresses, its goals, its activities, its resources and how the activities are expected to lead to results. This is often summarized in a logic model.
Example: Students meet with a mentor twice a week after school (activity), which is expected to increase their connection to school (short-term outcome) and improve attendance (long-term outcome).
Step 3: Focus the Evaluation Design
Decide what the evaluation will answer, what it will measure and how. Choose questions, methods and a timeline that fit the program’s purpose and resources.
Example: The main question is whether attendance improves. The evaluation will compare attendance before and during the program, and also track whether mentoring sessions happen as planned.
Step 4: Gather Credible Evidence
Collect information that stakeholders will see as trustworthy. Use reliable, valid measures, and consider more than one source of data.
Example: The social worker collects school attendance records, mentor session logs and short surveys from students and parents.
Step 5: Justify Conclusions
Analyze the evidence and interpret it against agreed-upon standards. Conclusions should be supported by the data, and should consider other possible explanations.
Example: Attendance improved for most participants, but the social worker also checks whether attendance rose schoolwide that year before crediting the program.
Step 6: Ensure Use and Share Lessons Learned
Share findings with stakeholders in ways they can use, and make sure results inform decisions about the program’s future.
Example: The social worker presents the results to staff and the funder, recommends expanding the program to a second school, and adjusts mentor training based on feedback.
Watch: In the video below, I walk through all six steps with a sample program to help you understand this concept completely.
[VIDEO: paste your existing video embed here]
3) Research Basics for the ASWB Exam
Program evaluation questions often overlap with general research questions. These are the concepts most worth knowing.
Reliability vs. Validity
| Reliability | Validity | |
|---|---|---|
| Means | Consistency: does the measure give the same results each time? | Accuracy: does it measure what it’s supposed to? |
| Types | Test-retest: same results over time Inter-rater: different raters agree Internal consistency: items on the measure relate to each other |
Face: looks like it measures the concept Content: covers all parts of the concept Criterion: matches an outside standard Construct: measures the intended theoretical idea |
A measure can be reliable without being valid, like a scale that’s always five pounds off. But it can’t be valid without being reliable. Learn more: Reliability vs. Validity and the ASWB Exam
Levels of Measurement
| Scale | What it is | Example |
|---|---|---|
| Nominal | Names or categories, with no order | Marital status, primary language |
| Ordinal | Ranked order, but the gaps between ranks aren’t equal | Mild / moderate / severe; satisfaction ratings |
| Interval | Equal gaps, but no true zero | Temperature in Fahrenheit |
| Ratio | Equal gaps and a true zero | Age, income, number of sessions attended |
Ask two questions: Are these categories, a ranking or numbers? Is there a true zero? Learn more: Research Scales and the ASWB Exam
Variables and Sampling
- Independent variable: the cause or intervention (the mentoring program).
- Dependent variable: the outcome being measured (school attendance).
- Random sample: everyone has an equal chance of being selected. Best for generalizing results.
- Stratified sample: random selection within subgroups, so each group is represented.
- Convenience and snowball samples: whoever is available, or participants recruiting others. Easier, but less representative.
Research Designs
| Design | What it does | Clue in the question |
|---|---|---|
| Randomized controlled trial (RCT) | Randomly assigns people to an intervention or control group. The strongest design for showing cause and effect. | “Randomly assigned,” “control group” |
| Quasi-experimental | Compares groups without random assignment | Two existing schools or clinics compared |
| Cross-sectional | Looks at one point in time | A single survey “this year” |
| Longitudinal | Follows the same people over time | “Followed for five years” |
| Case study | An in-depth look at one or a few cases | A detailed account of one client or program |
| Qualitative / quantitative / mixed methods | Experiences in words / numbers and statistics / both | Interviews and themes vs. scores and averages |
Learn more: Research Design and the ASWB Exam
Single-Subject Designs
Single-subject designs are the research designs social workers use most in practice: measure one client repeatedly to see whether an intervention makes a difference. In these designs, A is the baseline (no intervention) and B is the intervention.
- AB: baseline, then intervention. Simple, but it can’t rule out other causes for the change.
- ABA / ABAB: withdraws the intervention and, in ABAB, reintroduces it. This gives stronger evidence the intervention caused the change, but withdrawing help raises ethical concerns.
- Multiple baseline: starts the intervention at different times across behaviors, settings or clients.
Reading Results
- Mean, median and mode: the average, the middle value and the most common value. Use the median when there are extreme outliers.
- Standard deviation: how spread out the scores are.
- Statistical significance: a result unlikely to be due to chance, often defined as p < .05.
- Correlation: two things move together. Correlation doesn’t prove one causes the other.
4) Research and Evaluation Ethics
Ethics questions about research are common, and the NASW Code of Ethics (Standard 5.02) addresses them directly. Key principles:
- Voluntary, informed consent: participants understand the purpose, risks and benefits before agreeing.
- Right to withdraw: participants can stop at any time, without penalty or any effect on the services they receive.
- No coercion: services can’t depend on participating in research.
- Confidentiality: protect participants’ identities and data.
- Review: research involving people should be reviewed by an institutional review board (IRB).
5) Common Research and Evaluation Traps
- Counting outputs as outcomes: high attendance or many brochures doesn’t prove a program worked.
- Confusing reliability and validity: consistent isn’t the same as accurate.
- Calling an ordinal scale interval: ranked categories don’t have equal gaps.
- Treating correlation as causation: only a strong design, like an RCT, supports cause and effect.
- Pressuring participants: any answer that keeps a client in a study they want to leave is wrong.
- Mixing up process and outcome evaluation: process asks “Is it running as planned?” Outcome asks “Did it work?”
6) Practice Questions: Research and Program Evaluation
1. A social worker wants to test a new CBT group for adults with depression. Participants are randomly assigned to the new group or a control group that doesn’t receive it, and depression is measured before and after. Which research design is this?
A. Cross-sectional study
B. Case study
C. Randomized controlled trial
D. Longitudinal study
Answer: C. Random assignment to intervention and control groups, with before-and-after measurement, is an RCT. A longitudinal study follows people over time but doesn’t require a control group.
2. A survey asks for education level: less than high school, high school graduate, some college, bachelor’s degree or graduate degree. What level of measurement is this?
A. Nominal
B. Ordinal
C. Interval
D. Ratio
Answer: B. The categories have a meaningful order, but the gaps between them aren’t equal.
3. Two social workers independently rate the same recorded sessions using a checklist, and their ratings closely match. What does this demonstrate?
A. Inter-rater reliability
B. Test-retest reliability
C. Construct validity
Answer: A. Agreement between different raters is inter-rater reliability. Test-retest reliability would compare the same measure over time.
4. Three months into a new after-school program, an agency wants to know whether sessions are being delivered as designed and how to improve them. Which type of evaluation fits BEST?
A. Needs assessment
B. Outcome evaluation
C. Cost-benefit analysis
D. Process evaluation
Answer: D. Checking whether a program is delivered as planned while it’s running is process (formative) evaluation. Outcome evaluation asks whether the goals were met.
5. Midway through a study, a participant who is also a client at the agency tells the social worker she wants to stop participating. What should the social worker do?
A. Remind her that she agreed to complete the study
B. Let her withdraw without any effect on her services
C. Ask her to finish the final survey before withdrawing
Answer: B. Participation is voluntary, and participants can withdraw at any time without penalty or any effect on their services.
Want all of this on 3 printable pages? Download the free Research & Evaluation Cheat Sheet.
7) FAQs: Program Evaluation and Research on the ASWB Exam
Q: What are the 6 steps of program evaluation?
A: Engage stakeholders, describe the program, focus the evaluation design, gather credible evidence, justify conclusions, and ensure use and share lessons learned.
Q: What is the difference between formative and summative evaluation?
A: Formative (process) evaluation happens while a program is running, to improve it. Summative (outcome) evaluation happens at the end, to judge whether the program achieved its goals.
Q: Do I need to know statistics for the ASWB exam?
A: Not calculations. You should know basic concepts like mean, median and mode, statistical significance, and the difference between correlation and causation, and be able to apply them to a scenario.
Q: What is the difference between reliability and validity?
A: Reliability is consistency: a measure gives the same results each time. Validity is accuracy: a measure actually measures what it’s meant to.
Q: Which research design best shows cause and effect?
A: A randomized controlled trial (RCT), because random assignment to intervention and control groups rules out many other explanations for the results.
8) Conclusion
Research and program evaluation questions are less about math and more about judgment. Know the six steps of program evaluation, match the type of evaluation to the question being asked, measure outcomes rather than outputs, and protect participants’ rights. With the basics in this guide and some practice, these questions become some of the most predictable on the exam.
Get hundreds of practice questions with detailed rationales and a study plan to keep you on track with Agents of Change. We’ve helped more than 100,000 Social Workers prepare for their ASWB exams and want to help you be next!
Disclaimer: This content has been made available for informational and educational purposes only. This content is not intended to be a substitute for professional medical or clinical advice, diagnosis, or treatment.
————————————————————————————————————————————————
► Learn more about the Agents of Change course here: https://agentsofchangeprep.com
About the Instructor, Meagan Mitchell: Meagan is a Licensed Clinical Social Worker and has been providing individualized and group test prep for the ASWB for over five years. From all of this experience helping others pass their exams, she created the Agents of Change course to help you prepare for and pass the ASWB exam!
Find more from Agents of Change here:
► Facebook Group: https://www.facebook.com/groups/aswbtestprep
► Podcast: https://anchor.fm/agents-of-change-sw
#socialwork #testprep #aswb #socialworker #socialwork #socialworktest #socialworkexam #exam #socialworktestprep #socialworklicense #socialworklicensing #licsw #lmsw #lcsw #aswbexam #aswb #lcswexam #lmswexam #aswbtestprep #aswbtest #lcswtestprep #lcswtest #lmswtestprep #lmswtest #aswbcourse #learningstyles #learningstyle
Disclaimer: This content has been made available for informational and educational purposes only. This content is not intended to be a substitute for professional medical or clinical advice, diagnosis, or treatment
