Description
Making Afterschool
Programs Better
Denise Huang & Ronald Dietel
Policy Brief
No. 11
Copyright ? 2011 The Regents of the University of California.
The work reported herein was supported in part by a subcontract from Southwest Educational Development Laboratory (SEDL,
subcontract no. PO54604) funded by contract no. ED-03-CO-0048, as administered by the U.S. Department of Education.
The findings and opinions expressed in this report are those of the authors and do not necessarily reflect the positions or policies of
the National Center for Research on Evaluation, Standards, and Student Testing (CRESST), the Southwest Educational Development
Laboratory (SEDL), or the U.S. Department of Education.
To cite from this report, please use the following as your APA reference:
Huang, D., & Dietel, R. (2011). Making afterschool programs better. (CRESST Policy Brief). Los Angeles, CA: University of California.
The authors thank the following for their assistance with this policy brief: Joan Herman for her valuable review and feedback;
Tamara Lau for her design and layout; and Karisa Peer for her editorial support.
Introduction
________________________________________________________________________________________________________________________
The number of students attending afterschool programs has skyrocketed in recent years, currently
serving an estimated 8.4 million children (Afterschool Alliance, 2009). Consequently, the demand
for high quality afterschool programs, and to learn from successful ones, has never been greater.
This Making Afterschool Programs Better policy brief, synthesizes the results from nearly 20 years
of experience in evaluating afterschool programs. CRESST evaluations include U.S. Department
of Education supported 21st Century Community Learning Center programs, state-supported
afterschool programs across California, and multiple evaluations of the LA?s Better Educated
Students for Tomorrow enrichment program?.
CRESST is pleased to share this brief with the goal that others will benefit from our findings and
recommendations; thus, providing an increasing number of children with a high quality afterschool
program leading to exceptional learning and successful careers.
Key Components of
Effective
Afterschools
________________________________________________________
A growing body of research has found that students?
participation in afterschool programs is beneficial to
academic achievement and social adjustment (Pierce, Hamm,
& Vandell, 1999; Posner & Vandell, 1994). A recent study, for
example, found that students who participated in afterschool
programs had significantly higher reading achievement and
were rated by teachers as having a greater expectancy of
success than students who did not participate in afterschool
programs (Mahoney, Lord, & Carryl, 2005).
Other research has found that those students who participate
in quality afterschool programs the longest (both in terms
of frequencies and duration) have lower criminal activity
rates (Goldschmidt, Huang, & Chinen, 2007; Huang et. al,
2006; Lamare, 1997). Research has also discovered that
higher rates of participation in afterschool programs can
contribute to higher scores on academic standardized tests in
mathematics, reading, and language arts (Huang, Gribbons,
Kim, Lee, & Baker, 2000; Huang, Leon, La Torre, & Mostafari,
2008). Mu?oz (2002) found a positive relationship between
afterschool program participation rates and improved day
school attendance and academic achievement.
Drawing from research and more than a dozen afterschool
evaluations, CRESST developed a model for what we
consider five key components of effective afterschool
programs:
______________________________________________________________________________________________________________________
?Unless otherwise indicated, specific data in this policy brief are from CRESST Report 768, ?What Works? Common Practices in High Functioning Afterschool Programs Across the Nation
in Math, Reading, Science, Arts, Technology, and Homework?A Study by the National Partnership.? The report (as well as numerous afterschool evaluations) is available on the CRESST
website: www.CRESST.org.
National Center for Research on Evaluation, Standards, & Student Testing | Full Policy Brief No. 11 | Summer 2011
1
1
Goals are clear, rigorous, and supported across the program in structure and content. Funding
is adequate to support goals.
2
Leadership is experienced, well-educated, has longevity at the current site, uses effective
communications, sets high expectations, and has a bottoms-up management style.
3
Staff is experienced, has longevity at current program, relates well to students, models high
expectations, motivates and engages students, and works well with leaders, colleagues, and
parents.
4
Program aligns to the day school, provides time for students to study, learn and practice;
includes motivational activities, frequently uses technology, science and the arts to support
youth development, student learning, and engagement.
5
Evaluation uses both internal (formative) and external (summative) methods. Evaluative
information and data accurately measure goals; results are applied to continuous program
improvement.
These five components work together to produce a high quality afterschool program (Figure 1).
GOALS
LEADE RSH I P
QUALITY
AFTERSCHOOL
PROGRAM
STAFF
PROG RAM
EVALUATION
Figure 1. CRESST Afterschool Program Quality Model.
Making Afterschool Programs Better | Denise Huang and Ron Dietel
2
The remainder of this brief covers each of these five
components. First, we provide common evaluation findings
across most programs . We then provide examples,
either best practices or useful observations, noted by
the CRESST team . We conclude with a short list of
recommendations, which could help make afterschool
programs even better.
Component
1: Goals
________________________________________________________
?Everything that we do in afterschool, we try to make it
connected to the real world?.One of our many goals is to
make it relevant?make a connection for students to stay
in school, go to college, a reason to be on a particular
career path?? – Afterschool Project Director
Overview
Setting clear goals and desired outcomes is a cornerstone
of afterschool program success (Bodilly and Beckett, 2005).
Virtually all of the programs we studied had established
unambiguous goals and structured their programs to
meet those goals. At the same time, most programs also
recognized the importance of considering student voices
when making decisions regarding program activities
and content; hence, many programs allowed students
to provide input, especially in the arts and technology
programs. As a result, students were engaged and excited
to be in their afterschool programs.
Key Evaluation Findings
The best programs had:
? clearly defined goals in a written plan;
? curricular design and specific practices aligned to
program goals; and
? internal and external evaluations (for further details,
see Component 5 of this report).
?Achievement? focused programs, often districtaffiliated, were:
? usually more structured than other afterschool
program types;
? stressed improvement of academic performance;
? hired more certified teachers; and
? maintained a more consistent linkage with the day
school then less achievement-focused programs.
Goals of many high quality afterschool programs also
had a specific emphasis (e.g., science, technology,
homework support, community involvement, or the
arts).
Examples
The site coordinator at one math program described
program goals as focused on developing students?
academic skills within a specific content area: ?Our
primary goals are to bring the student to grade level.?
Similarly, goals for three of the math programs and four
of the reading programs specifically targeted students
who were struggling academically.
A primary goal of many arts focused afterschool
programs was to provide students, who otherwise
would have little-to-no exposure to the arts, with
quality art experiences. Many interviewees reported
using an integrative approach to arts instruction that
could help students who were struggling academically
and personally.
Similar to arts, all science staff responded to questions
about curricular goals by suggesting a primary interest
in offering positive science experiences to students. A
few went further to explain that their principle goal
was to improve standardized test scores in science
by focusing on extending day school instruction into
afterschool.
Interview data across seven homework-focused
programs suggested that the primary goals in
afterschool homework were to complete homework
and increase academic achievement.
Nearly all of the technology staff had the goal of
teaching students the mechanics of a broad range
of technology skills, an interest in encouraging the
students to use those skills to enhance learning in other
National Center for Research on Evaluation, Standards, & Student Testing | Full Policy Brief No. 11 | Summer 2011
3
100%
90%
Percentage of Instructors
80%
70%
60%
50%
40%
22%
30%
20%
17%
19%
14%
28%
10%
0%
Never
Rarely
Sometimes
Frequently
Regularly
Figure 2. Frequency of communication between afterschool program and day-school staff.
content areas, and teaching the students a technical skill
that is relevant to their real lives (both now and in the
future).
In aligning activities to achieve their program goals, nearly
all interviewees described the merits of developing a
curriculum that combined academic skill development with
opportunities to explore and encourage students? social
development. One program director summed up:
?Our goal is to help each child to make sure that it?s
an afterschool program that?s fun, but at the same
time it?s enriching, so they grow and learn every day,
so they can take home more knowledge??
Component
2: Leadership
________________________________________________________
?Obviously, we try to be democratic?.So one of the
things we try to do here, how we want to make this
a great place to work, is in finding great people,
then giving the people the power and leeway and the
accountability, but also the freedom to do what they
think works best, and trusting them?.?
– Afterschool Project Director
Overview
Our studies found that directors and managers of high
functioning afterschool programs usually had many years
of experience in afterschool programs. Further, the leaders
nearly always shared decision-making with their staff across
afterschool goals, programs, and evaluation.
Key Evaluation Findings
The best programs had leaders who:
? articulated a clear program mission, vision statement,
and goals;
? decentralized decision making; used a bottoms up
leadership style;
? trusted in the knowledge and skills of staff regarding
curriculum and instruction;
? promoted a team culture of positive relationships,
frequent communication, and staff problem-solving
skills;
? insured that instructors and students had adequate
materials and resources; and
? provided all staff with professional development
opportunities that improved individual and team skills.
The majority of both site coordinators and instructors
said that afterschool instructors had an active voice in
decisions about curriculum and instruction (4.4 average
on a 5.0 point scale) and took active roles in program
leadership and decision-making (4.1 average).
Administrators consistently described the value of
staff?s content-specific skills and expertise; as a result,
curriculum development was a group process in which
leaders gave staff members a strong voice in designing
instruction.
Leaders and staff across high-quality programs
maintained good relationships with the day school
personnel. However, few of them had established
formal communication systems for that purpose
Making Afterschool Programs Better | Denise Huang and Ron Dietel
4
24%
Master?s Degree
1%
Currently in High School
19%
High School Graduate/GED
47%
Bachelor?s Degree
9%
Associate?s Degree
Figure 3. Educational levels of afterschool staff.
(as indicated in Figure 2), with less than 50% of
afterschool staff reporting that they had regular or
frequent communication with day school teachers (see
Recommendations section).
Examples
One site coordinator expressed appreciation for the staff
by stating,
?These are professional adults?and they are the
best ones to implement the curriculum? They?re
right there with the students?they know what
their levels are and their abilities.?
An arts program instructor explained,
?My experience has been that the afterschool
teachers propose something that we?d like to do
with the kids afterschool and [the director] then
just talks to us about what our plans are. We?
report to her in terms of lesson plans and how we
incorporate standards and benchmarks, but a lot of
freedom is really given to us. We teach what we?re
comfortable teaching and what we?re passionate
about.?
A project director at a technology program said that his
staff?s latitude was evenly balanced by a strong level of
accountability for their curricular choices.
?Every quarter they have to come back to us and
tell us how they?re doing. They report back as
to what is going on at their?programs. In terms
of actual decision-making and setting goals and
deciding what we?re going to do, that?s much more
of a bottom-up process.?
Component
3: Staff
_____________________________________
?We recognized that the other role we (staff) have to
play is to get children engaged in the learning process.
So it?s not just about completing the homework, but it?s
about finding ways outside of the school to get them
interested, excited, feeling confident, and to build their
self-esteem, so that they want to come back the next day
and try a little harder.? – Afterschool Staff Instructor
Overview
We found that high quality programs recruited qualified
staff and created collegial environments supporting their
programs? missions. Afterschool leaders were able to retain
staff and achieve lower turnover rates than other programs
because staff felt respected, supported, autonomous, and
confident in their ability to reach their students. In turn,
staff and students constructed positive relationships with
each other, characterized by warmth and mutual respect.
Staff was often role a model for students, creating a norm
of high expectations, appropriate student behavior, good
school attendance, effective work habits, and positive
attitudes towards learning.
Key Evaluation Findings
Staff from quality programs generally had high
educational levels. Forty-seven percent of all staff had
Bachelor?s degrees; twenty-four percent had a Master?s
degree (see Figure 3).
National Center for Research on Evaluation, Standards, & Student Testing | Full Policy Brief No. 11 | Summer 2011
5
Afterschool staff in quality programs usually had
substantial afterschool experience. Sixty-six percent
had more than three years of work experience in
afterschool; twenty-three percent had six years or more
of experience.
Program staff at high quality sites also tended to
have low turnover. The majority of the staff had
been at their current site for three years or more (see
Recommendations section).
Positive relationships and interactions between the staff
and the students were observed in virtually all high
quality programs, particularly in terms of expectations
for student performance, disciplinary issues, and
democracy.
All interviewees reported having some form of
technology-related professional development available in
their afterschool program.
Most professional development opportunities were for
helping students with reading and math, applying state
standards to the curriculum, and connecting with the
day school. Fewer opportunities were offered on topics
of English learners, special needs students, evaluation,
and assessment (see Recommendations section).
A higher percentage of site coordinators reported
receiving professional development in most categories
(other than working with English language learners),
compared to instructors who reported receiving the
lowest percentage of professional development in
program and self-evaluation (see Recommendations
section).
Although staff generally found professional development
useful, only 26% had regular (2-4) opportunities each
year. About 50% of the homework program staff
reported that no professional development was offered
to them (see Recommendations section).
Examples
Nearly all quality afterschool programs approached
decision-making in a democratic fashion. One program
allowed students to offer input on where they would
like to conduct their service learning projects; others
considered students? activities choices and made great
efforts to include them in their program decisionmaking.
Arts-, science-, and technology-focused afterschool
programs tended to give students more autonomy
and input into programs compared to homework or
academic focused programs. One arts program staff
stated that student inputs were ?always of great value;
student interests had a vast impact on art curriculum
content.? Consequently, students took ownership in
their learning and remained engaged.
Component
4: Staff
____________________________________________________
?These are kids who have never been exposed to
anything?our goal is to open their minds to new
things, and to show that they can do it?.We want a
well-balanced, well-rounded program with a lot of
different things to offer to the children.? – Afterschool
Site Coordinator
Study findings revealed curricular similarities and
differences across all programs, a majority of which
offered three or more activities each day. Most
programs included homework help or tutoring, but
other activities ranged from academic (e.g., math,
reading, writing, science) to enrichment (e.g., arts
and crafts, cooking, gardening, health and nutrition,
cultural activities, computers) and recreation (e.g.,
sports, dance, drill team, outdoor games). The
frequency and duration of instruction offered by the
programs are provided in Table 1.
These findings suggest that students were receiving
adequate afterschool time for learning and skills
practice. Observation reports across the programs also
indicated that students appeared to be mostly engaged
and attentive, and enjoyed the activities. Virtually all
programs provided substantial time for recreational
and crafts activities, keeping students engaged while
exercising other parts of their brains.
Key Evaluation Findings
The majority of programs employed unique and
innovative strategies to engage students in the
afterschool setting, placing a particular emphasis on
making learning fun.
Technology programs reported the most frequent
use of research-based practices, whereas reading
programs reported the lowest frequency.
Technology, science, arts, and homework programs
appeared to be more focused on developing
higher order thinking skills, whereas reading and
math programs were more focused on direct skills
instruction.
Most programs shared similar methods of
disseminating information to parents, as well as
a means of encouraging their involvement in the
afterschool programs
Parents were very satisfied with the programs both
in terms of positive changes in their children?s
behaviors and attitudes, and in general program
functioning. They felt that the staff cared about and
respected their children. They also reported that
Making Afterschool Programs Better | Denise Huang and Ron Dietel
6
Content area
Average duration of
daily instruction
Average number of days
offered per week
Reading
51 min.
3.20
Math
66 min.
2.58
Science
77 min.
2.28
Art
77 min.
3.72
Technology
105 min.
3.55
Homework
45 min.
4.00
Table 1. Duration and Frequency of Instruction by Content Area
afterschool staff dealt with their children?s behavioral
problems promptly.
Over half of the programs took field trips to enhance
student learning and motivation.
Many of the high quality programs had social or
character development in their curriculum as well as a
community involvement component.
However, links to day school curriculum were at low
levels, even at high quality sites. Communications
with the day school teachers occurred mostly in forms
of brief, informal contacts such as email or casual
drop-in conversations. The key topics on most of the
communications between day school and afterschool
were homework related (see Recommendations section).
The need for additional space was commonly mentioned
by both site coordinators and program staff (see
Recommendations section).
According to most program staff, another scarce
resource was access to technology and particularly
computers (see Recommendations section).
Almost all programs had fairly low parent participation
and volunteerism, as well as low percentages of formal
parent?teacher meetings. For example, only 19% of
staff met with parents on a regular basis. Low parent
involvement was usually attributed to parental work
commitments (see Recommendations section).
Examples
Afterschool programs used a number of strategies to
keep students engaged in learning, including cross-
content integration, diversity of activities, real world
examples, dialogic and cooperative learning, and
cultural awareness programs. Learning was often
embedded discretely in sports, games, discussions,
and journal writing. A site coordinator explained the
approach:
?I think that because a lot of our program is
disguised learning, a lot of times the kids don?t
even realize that they are doing math or that they
are doing language arts or reading.?
Staff at one site described a science program that
was the result of a collaborative effort between day
school and afterschool instructors, designed around
the district?s science curriculum, grounded in state and/
or national science standards, and supplemented with
purchased science kit materials. Science lessons revolved
around a fish hatchery theme, developed primarily
during day school instruction, and maintained and
studied throughout the year in both the day school and
the afterschool program. According to interviewees, this
concept provided a focus that gave students a beneficial
sense of continuity between day school and afterschool
science instruction.
Almost all afterschool programs offered arts activities
for their students as enrichment including poetry,
dance, drama, choir, and drawing. One program
instituted an innovative program, ?Fun with Junk,?
where kids created art projects out of recyclables. Other
sites put on drama, dancing, or singing productions
for fellow students, teachers, and parents; thereby,
providing opportunities for students to collaborate with
each other.
National Center for Research on Evaluation, Standards, & Student Testing | Full Policy Brief No. 11 | Summer 2011
7
?
Many sites provided character development. The site
coordinator at a reading afterschool program described
two character programs that they offered students:
The girls get ?Smart Girls?, which is also character
building but it deals with those life changes during
puberty, taking care of your body in terms of
hygiene, and what does it mean to be a woman in
society. For the boys it?s ?Passport to Manhood.?
Another site coordinator for a math afterschool program
mentioned a program called Character Development,
which focused on teaching students values, such as
honesty, respect, responsibility, and caring.
Community involvement was common in the programs,
such as making get well cards for patients in the
hospitals, taking trips to nursing homes to sing to the
senior citizens, recycling or community beautification
efforts, and cultivating community gardens. One
afterschool program worked with a local charitable
organization not only to restore a public garden, but
also to hike, take horticulture classes, go rowing, and
swim in the lake.
?
Community members often volunteered in the afterschool programs, often a result of affiliations with local
universities and high schools, who frequently tutored
students. Boy and girl scouts, churches, and boys
and girls club members also volunteered. Afterschool
program staff frequently invited science experts from
the community to visit their programs and share their
own experiences of practical, real-world applications for
science. As one project director explained,
?What makes it unique [at our program] is we
have so much community involvement in teaching
science?We?ve really tried to get experts in the
field to come in?I don?t think that there is any
program that has more community involvement in
teaching students science than ours.?
Component
5: Evaluation
________________________________________________________
When you look at their assessments on the [state
test]?they weren?t measuring up with other states;
and because of that we had to go back and revisit our
curriculum to see where we were falling through the
cracks. – Afterschool Program Principal
Overview
As outlined in the CRESST afterschool model, ongoing
evaluation is necessary to measure program performance
and make continuous improvement. While evaluation data
serves many stakeholders including students, parents, and
afterschool staff? funding agencies (who are making a
growing investment in afterschool programs) increasingly
want to know if their outlays are making measureable
improvements.
We found that nearly all afterschool programs used
internal evaluation to identify program strengths and
weaknesses. Internal evaluation, often called formative
evaluation, was usually done by the program?s own staff or
staff within its funding agencies governance structure. The
stakes or consequences were usually low ? with program
improvement the key goal. An external evaluation,
on the other hand, typically had higher consequences
for programs and was nearly always conducted by a
disinterested third party. In some cases, accreditation or
even program continuance may be decided by an unbiased
outside, ?external? evaluation expert.
Interview and survey responses across our studies indicated
that even though rigorous examination of data was rare,
nearly all of the afterschool programs conducted internal
or external evaluations of their programs.
Key Evaluation Findings
All but two programs reported having conducted
internal evaluations. Evaluation varied from informal
conversations between afterschool staff, day school
staff, and parents, to a formal administration of surveys
to students, parents, staff, and tracking of test scores,
grades, and attendance records (see Recommendations
section).
Responses from interviewees suggested that many of
the programs were evaluated externally, sometimes
by an evaluation organization experienced in program
evaluation. Interviewees consistently indicated that most
of the evaluations were of the entire program.
External evaluation methodologies typically included
pre-post testing or classroom evaluations, comparison
groups, surveys, focus groups, observational
assessments, or a combination of methods.
Making Afterschool Programs Better | Denise Huang and Ron Dietel
8
In general, interviewees from the majority of the
programs reported positive results from external
evaluations, although specific results were often not
substantiated with reports (see Recommendations
section).
For the reading and math programs that were closely
affiliated with school districts, approximately one third of
the programs mentioned having an external evaluator.
Examples
In general, interviewees most frequently mentioned
using the results of internal evaluation to serve as a
baseline for instruction, monitor student progress, and
document program impact. For example, the principal at
one of the science afterschool programs reported that
student achievement data from an internal evaluation
were used to revise science curricula at the afterschool
program in order to align instruction with the standards
and improve student performance.
A few formal external evaluations and systematic
tracking of student progress showed improvements in
attendance, classroom grades, and achievement scores.
About half of the afterschool staff reported that the
students in their afterschool programs had improved
their day school attendance, were paying more
attention in class, and had fewer discipline problems
(see Figure 4).
Fifty eight percent of teachers agreed that the
afterschool program students increased their frequency
of class participation; most teachers (61%) agreed that
their students in the afterschool program put more
effort into school work.
At the less quantifiable level, several programs reported
improvements in attitudes toward schooling, student
efficacy, confidence, and engagement.
?Attendance is unbelievable. I have kids that say,
?I only came to school today because I knew I was
going to be working with you.? I feel, just from
talking with my teachers, that behavior problems
in some instances are resolved. Students have
success in my class.?- Afterschool Arts Instructor
100%
90%
80%
Percentage of Teachers
70%
60%
50%
40%
30%
20%
10%
0%
School Attendance
strongly disagree
Frequency of
class participation
somewhat disagree
Effort on
School Word
neither disagree or agree
Paying Attention
in Class
somewhat agree
Fewer
Discipline Issues
strongly agree
not applicable
Figure 4. Teacher perception of changes in student behavior.
Making Afterschool Programs Better | Denise Huang and Ron Dietel
9
Recommendations
________________________________________________________________________________________________________________________
Our findings point to the contributions of goals, leadership, staff, programs and evaluation to high quality afterschool
programs. Nevertheless, we noticed consistent areas of weaknesses in even the best afterschool programs. We offer the
following research-based recommendations
that we believe will help improve any afterschool program:
Staff Stability
Our study?s results indicated that high functioning programs tend to have low staff turnover rates. Over 60%
of the staff had between 1?7 years of experience at the current site and over 30% of the staff had over 4 years
of experiences at the current site. Staff stability is important for relationship building and a basis for students to
build trust, positive attitudes, and efficacy toward learning.
Recommendation. Funders and afterschool administrators should consider incentives for building staff retention, ranging from educational opportunities (e.g., tuition grants) to ?outstanding afterschool teams or teachers? nominated and
selected by parents, teachers, and students. A pay scale incentive for years of service and a possible career ladder may also
improve staff retention.
Collaboration with the Day School
In our studies, all of the programs maintained positive relationships with the day school. However, despite
the importance of this relationship, too few programs had strategic systems established that supported and
strengthened this connection.
Recommendation. Formal agreements and written plans (ideally in early stages) should address day school
collaboration. Time for day school teachers and afterschool staff to meet and plan lessons together plus a communications
system, (e.g., homework log between day school teachers and afterschool staff), should be included in both school and
afterschool plans. Funding agencies should budget additional resources for afterschool programs that will facilitate linkages,
such as shared professional development, staff retreats, or workshops that jointly support students.
Space and Technology
Many programs relied on access to common space, such as an auditorium or a classroom shared with day
school teachers, which often caused logistical problems and sometimes prevented planned activities from taking
place. Furthermore, some programs expressed difficulty with not having consistent access to classrooms. A
site coordinator illuminated the problem, ?I would say physical space would be definitely a big thing with our
program?.That is probably one of the hardest things to work with just because every 6 weeks we are readjusting
the classroom to new classroom seating charts, new areas in which the students can and cannot go.?
?
Recommendation. Afterschool space issues should be addressed early in the planning process and reviewed each
year for adequacy. Technology too, especially with shared equipment, should also be addressed, recognizing the growing role
that technology plays in both learning and recreational activities.
?
Formal agreements
and written plans,
ideally in early stages,
should ad
