|
ACHIEVEMENT IN YEAR-ROUND SCHOOLS Bradley J. McMillen North Carolina Department of Public Instruction Division of Accountability Services 301 North Wilmington Street Raleigh, NC 27601 Email: bmcmille@dpi.state.nc.us (919) 807-3808 (Under Review - Journal of Educational Research) The author thanks Carolyn Cobb, Kris Kaase and Gongshu Zhang for their feedback on design and analysis issues.
Existing evidence implies that achievement in year-round schools is either equal or superior to that of traditional calendar schools, and that its positive effects may be particularly strong for more disadvantaged students. However, most studies on this topic suffer from serious methodological limitations. The purpose of this study was to better examine achievement differences between year round and traditional calendar students using two years of data from over 345,000 North Carolina public school students in grades 3 through 8. Results indicated that achievement in year-round schools was no higher than in traditional calendar schools, and that the differential effect for disadvantaged students, although statistically significant, was not of practical significance. Time has long been a variable of interest in countless educational contexts. A report by the National Education Commission on Time and Learning (1994) discussed at length the importance of time in education, the extent to which time controls what happens in schools, and the need for more efficient use of time to promote greater student achievement. One of the major conclusions in this report called for more time devoted to instruction and learning in core subject areas. This issue of quantity is also addressed by Carroll's (1963) model of school learning, which designates the amount of instructional time as one of the more malleable factors in determining student learning. Subsequent international comparative investigations such as the TIMSS studies (Martin, Mullis, Gonzales, Smith, & Kelly, 1999) have shown that countries where students spend more days in school and more time during the day on mathematics and science instruction demonstrate higher achievement in those subject areas. The sheer quantity of instructional time that is available to students is undoubtedly critical in determining student outcomes, and the variety of educational policies concerning the numbers of credit hours required to earn a diploma or degree and the number of instructional hours in a school year are testament to this relationship. In addition to quantity, other studies have addressed the issue of the quality of time as it relates to student learning. Research on time on task and academic learning time (Berliner, 1990; Fredrick & Walberg, 1980) have focused on the relationship between the amount of time students spend engaged in academic activities and how much they learn. Studies in this area have demonstrated that simply exposing students to classrooms and teachers is not sufficient to affect learning, implying that the educational quality of the activities and interactions that occur in those settings mediates the relationship between time and learning. Other studies of time and learning have focused on how academic performance varies by time of day, although it is not conclusive as to which time of day is better. For example, spelling and memory abilities appear to vary according to the time of day, although some types of errors seem to occur more often in the morning, while other types occur more often in the late afternoon/evening (e.g., Dunne, Roche, & Hartley, 1990; Folkard, Monk, Bradbury, & Rosenthall, 1977; Morton & Diubaldo, 1995). Other research has indicated that nonmedicated students with attention deficits have better problemsolving abilities and engage in less off-task behavior in the morning (Zagar & Bowers, 1983). Year-Round Education One of the most common debates about time and learning in recent decades has centered around the adoption of year-round school calendars. According to the National Association for Year-Round Education (NAYRE), the number of year-round schools operating in the U. S. has increased from just over 400 in the late 1980's to 2,880 during the 1999-2000 school year (NAYRE, 2000). Although it exists in many different forms, year-round education essentially involves the reorganization of the traditional school calendar so that the long summer vacation is replaced by several smaller breaks evenly spaced throughout the year. Although "year-round" is often used to describe "extended-year" schools (i.e., more than 180 days of instruction per year), the term "year-round" is used here to describe schools that have simply redistributed their 180 instructional days more evenly throughout the calendar year (Worthen & Zsiray, 1994), and its definition is summed up by the National Association for Year-Round Education (NAYRE, 2000):
The proliferation of year-round schools at the national level is also evident in North Carolina, where the number of year-round schools has grown from 73 in 1994 to 121 in 2000. There are two basic models of year-round education being utilized in North Carolina - school wide (SW) and school-within-a-school (SWS). In the SW model, all of the children in a school attend on a year-round (12 month) calendar. In the SWS model, one group of students attends on a year-round calendar, while the others attend on a traditional nine-month calendar. Teachers in the SWS model are also usually divided into two groups - those who teach in the traditional program, and those who teach in the year-round program. The SWS model basically creates one traditional August-May program and one year-round program, each with its own teachers and students, but operating on the same campus with the same administrative staff.
Although there are many types of
calendar arrangements in North Carolina's year-round schools, the most common is
one where students are in school for 45 days and then on break for 15 days.
Approximately 94% of year-round schools in North Carolina were on this schedule
during the 1998-99 school year (North Carolina Department of Public Instruction
(NCDPI), 2000). Although most SW year-round schools in North Carolina are
single-track, approximately 13% operate as "multi-track" schools. In
these schools, different groups (i.e., tracks) of students begin the school year
at different times, with the vacation schedules for each track distributed so
that at least one track of students is always out of school. Multi-track
year-round schools are often implemented to ease overcrowding because the school
building can then accommodate more students than if they all attended at the
same time. Several reviews of the existing literature on year-round education have been conducted, with the general consensus being that the outcomes of year-round education are at least as positive as (or better than) those achieved under the traditional school calendar. However, the number of quality studies conducted and published in this area is limited (Kneese, 1996). In one of the earliest reviews, Merino (1983) found no differences in achievement between students in year-round and traditional schools. Worthen and Zsiray (1994) later reevaluated the evidence on the effectiveness of year-round education based on several studies that focused on achievement, cost, satisfaction and other outcomes. They concluded that: (a) Achievement in year-round schools is equal to or greater than achievement in traditional schools; (b) teachers and students in year-round schools have more positive attitudes; (c) although there will always be some parents who do not like year-round calendars, most parents will be satisfied with a year-round program if it is well-implemented; and (d) single-track programs will cost as much or more than traditional school programs, while mufti-track programs can result in significant cost savings if those programs are well-implemented. Kneese (1996) conducted a similar review of 15 studies that focused specifically on achievement in traditional calendar and year-round schools. She concluded that achievement in year-round schools appears to be slightly higher than in traditional calendar schools. This finding is also echoed in other recent studies (Gandara & Fish, 1994; Shields & Oberg, 1999). Although researchers have not adequately addressed the reasons why achievement may be slightly higher in year-round schools, one possibility is that year-round schools can use intersessions to provide remediation and enrichment activities, thereby increasing students' exposure to the curriculum (Ballinger, 1995). Another possible explanation comes from a body of research that points toward a decline in achievement during the long summer vacation associated with the traditional school calendar (Cooper, Nye, Charlton, Lindsay & Greathouse, 1996; Entwistle & Alexander, 1992; Parsley & Powell, 1962; Wintre, 1986). Year-round advocates claim that dividing the long summer vacation period into smaller pieces helps alleviate some of the "forgetting" that occurs over the summer in traditional school programs. However, there is currently no specific scientific evidence to support that contention (Mitchell & Mitchell, 1999). There is also some evidence that year-round programs may be particularly beneficial for cognitively and economically disadvantaged students, especially in reading (Cooper et al., 1996; Handleman & Harris, 1984; Gandara & Fish, 1994). Despite the existence of several studies on the topic, most existing research on year-round education and achievement suffers from important methodological limitations. These include (a) failure to take student-level factors into account in the estimation of achievement effects, (b) a loss of precision in the dependent variable due to collapsing achievement outcomes into categories such as "at/below grade level" (e.g., Shields & Oberg, 1999), (c) failure to report any tests of statistical significance or measures of effect size (e.g., Alcorn, 1992), and (d) failure to differentiate between year-round and extended-year schools (e.g., Gandara & Fish, 1994). In addition, all of the studies in this area have failed to account statistically for the nesting of students within schools in their analyses, and few have been subjected to the peer review process. The goal of this investigation therefore was to provide a more statistically appropriate examination of achievement in year-round and traditional calendar schools.
Based on several archival data sources, 106 public schools were identified in North Carolina that operated on a year-round calendar in grades 3 through 8 during the 1997-98 school year. Analyses were conducted comparing the academic achievement of students who attended these year-round schools and programs during the 1997-98 school year to that of students who attended traditional calendar schools and programs. Data Source The data for the study were obtained from historical databases from the North Carolina Testing Program. Over 95% of public school students in North Carolina participate in this statewide testing program each year (NCDPI, 1999a). Testing Program data in grades 3 through 8 in North Carolina are gathered from end-of-grade (EOG) assessments in reading and math, and are reported in the form of normed developmental scale scores. With respect to content, EOG tests are closely aligned with the North Carolina Standard Course of Study, which is the official state curriculum. Test scores are scaled separately for each grade level and each subject area, and the normative distribution of scores in each subject area shifts upward slightly from one grade level to the next. Therefore, a student's scale score is expected to increase naturally from one grade level to the next. Due to this feature, scale scores cannot be directly combined or compared across multiple grade levels, because the distribution of possible scores is not in the same scale location from one grade level to the next. Since the current study used data from students across multiple grade levels, it was necessary to convert the scale scores to standard scores before any analyses could be conducted. Scores were standardized separately for each subject area by grade level combination. The resulting scores had a mean of 50 and a standard deviation of 10. In addition to achievement test scores, the Testing Program databases also contain demographic information collected from each student. Using this data, some student-level variables were created to be used as covariates in the achievement analyses. Although the actual focus of the analyses was to examine differences based on the year-round/traditional calendar conditions, these variables were included to provide a more precise test of differences between calendar conditions and to help rule out competing explanations should any differences between groups be found. The covariates used were the same for both the SW and SWS schools in both the reading and math and included each student's previous year EOG test scores, gender, ethnicity, and parent's highest level of education. Sample Selection Procedures
Since EOG tests are given only in grades 3 through 8, and because the study was
designed to look at achievement growth from one year to the next, the study used
all of the available data from students who took EOG tests in grades 4 through 8
in 1998. The final sample included all North Carolina public school students who
took EOG tests in either reading or math in 1998 at a given grade level and who
also took the EOG test in that same subject area in 1997 at the previous grade
level. Therefore, students who did not have available data in a given subject
area for both years were excluded from the analysis for that subject area.
Students who were retained during the 1997-98 school year were excluded from all
analyses.
Demographic information for the samples used in the reading and math analyses
are reported in Table 1. These data are based on student characteristics as
reported in North Carolina Testing Program databases for the 1997-98 school
year. Because some students only had test data available in one of the two
subject areas, there are some small differences between the reading analysis
sample and the math analysis sample. Demographic information is reported
separately for four groups of students: (a) students from schools that operated
a SW traditional calendar (T - SW), (b) students from schools that operated a SW
year-round calendar (YR SW), (c) students who attended on the traditional
calendar in a school that was operating a SWS model (T - SWS), and (d) students
who attended on the year-round calendar in a school that was operating a SWS
model (YR - SWS). In total, data from over 345,000 students from 1,470 schools
serving grades 3 through 8 were included in the analyses. The breakdown of the
number of schools by type is given in Table 2.
The four groups of students were very similar with respect to gender. However, there were patterns of differences between the four groups with respect to grade level, ethnicity, and parental education level (i.e., highest level of education attained. In both the SW and SWS model schools, students attending on a year-round calendar were concentrated at the lower grade levels, likely due to the fact that most year-round programs in North Carolina are in the K-5 range (NCDPI, 2000). In schools operating a SW model, traditional calendar students were more likely to be Caucasian and less likely to be African-American than year-round calendar students (Reading x2 = 935.25, p < .05; Math x2 = 934.26, p < .05). Also, year-round calendar students reported slightly higher levels of parental education (Reading x2 = 549.53, p < .05; Math x2 = 544.76, p < .05).
In schools operating a SWS model, however, the pattern of ethnicity differences
was reversed. In these schools, year-round programs consisted of more Caucasian
students and fewer AfricanAmerican students (Reading x2 = 43.53, p < .05;
Math x2 = 46.66, p < .05). As in the SW programs, year-round students
reported slightly higher levels of parental education (Reading p<.05;Math x²=15.0 Hierarchical linear modeling procedures were used to examine potential differences in achievement between the four groups of students. Traditional regression methods require either aggregating data to the school level prior to analysis, which results in a loss of statistical power and precision, or disaggregating school-level data down to the individual student level, which often results in spuriously significant results that show relationships between variables which may not truly exist (Hox, 1995). Hierarchical linear modeling methods avoid both of these problems by properly incorporating both school-level and student-level factors in the same analysis (Bryk & Raudenbush, 1992; Singer, 1998). These analyses were conducted separately for SW schools and SWS schools using the MIXED procedure from the SAS® statistical software package. School-Wide School Results The first set of analyses compared students in school-wide year-round schools to students in school-wide traditional schools. The initial model for each subject area used year-round status as the only predictor variable. The overall test of the calendar effect was not statistically significant either for reading (t = .68, p < .50) or math (t = .63, p < .53). After the covariates were added to the model, there was still no difference between students in year-round schools and students in traditional calendar schools with respect to either reading or math achievement (Table 3). The adjusted and unadjusted mean achievement scores for year-round and traditional school students are displayed in Figure 1.
School-Within-a-School Results The analyses within each subject area for the SWS programs also consisted of an initial model using year-round status as the only predictor variable. In the SWS analyses, however, there were two levels of nesting: students within year-round conditions within schools. As was true for the SW analyses, the overall test of the calendar effect was not statistically significant for either reading (t = -1.01, p < .32) or math (t = -.82, p < .42). After the covariates were added to the model, there was still no difference in achievement between students in the year-round track and students in the traditional track (Table 4). The adjusted and unadjusted mean achievement scores for year-round and traditional track students are shown in Figure 2.
Differential Effects Since previous research has raised the possibility that more disadvantaged students might benefit more from year-round education (Cooper et al., 1996; Gandara & Fish, 1994; Handleman & Harris, 1994), a third regression model was generated for both SW schools and SWS programs examining potential interactions between year-round status and prior achievement. These models were identical to the previous models with the student-level covariates (Tables 3 & 4) except for the addition of one interaction term.
The models for SWS programs in reading and math indicated no statistically
significant interaction between year-round status and prior achievement. The
models for SW schools, however, revealed statistically significant interactions
for both reading (t = -5.23, p < .05) and math (t = -2.44, p < .05).
Specifically, the lower-achieving students in year-round schools demonstrated
higher achievement than their traditional school counterparts (Figures 3 &
4). This relationship was slightly stronger in reading than in math, but in both
cases, the differences were not large. The differences among students in the
lowest achievement quartile, although statistically significant, amounted to
approximately .05 standard deviations in reading and even less in math.
Initial analyses indicated no statistically significant differences in either reading or math achievement between students attending school on a year-round calendar and those attending on a traditional August-May calendar during the 1997-98 school year. After controlling for a variety of student demographic factors as well as prior achievement, there were still no statistically significant differences between the two groups of students. These results were consistent for students in school-wide schools (i.e. schools in which all children attend on the same calendar) as well as students in schools-within-schools (i.e., schools which have a traditional and a year-round program on the same campus). For school-wide schools, a small interaction was found between prior achievement and yearround status, which implied that lower-achieving students may benefit from being on a year-round calendar. This effect was not found for school-within-a-school programs, however. The results of the current study are somewhat consistent with previous studies indicating that achievement of students in year-round schools is equal to that of students in traditional schools, and that year-round calendars may be particularly beneficial for lower-achieving students: Differences between the results obtained in this study and those which have found higher overall achievement in year-round schools may be due to methodological differences in data analysis and different definitions of year-round schools. For example, Gandara and Fish (1994) report achievement benefits for students in year-round schools. However, the year-round schools in that study actually had more than 220 instructional days in a calendar year, leaving open the question of whether those gains might have been attributable to the added number of instructional days as opposed to the use of a year-round calendar. Similarly, Shields and Oberg (1999) found achievement to be slightly higher in multi-track year-round schools; however, all analyses were conducted at the school level and failed to take into account any student-level factors which may have been related to year-round status and which may also have affected observed achievement. Although the inclusion of covariates associated with achievement is not a cure for non-random assignment, it does provide a stricter test of the potential effects of the year-round calendar on achievement. Given that many year-round programs are magnet programs and therefore may draw students from outside of the school's normal attendance zone, the consideration of student-level covariates in studies of year-round schools is essential. Not only do studies of year-round education often suffer the limitations inherent in retrospective non-experimental research, but local decisions about where year-round calendars are implemented and the potential effects of school choice may also produce systematic differences between students in year-round and traditional calendar schools and programs. For example, all SWS year-round programs and 69% of SW year-round schools in North Carolina are schools of choice (NCDPI, 1999b), which may have some bearing on the demographic differences between the traditional and year-round students in the current study. Other studies focusing exclusively on multi-track programs have also documented substantial differences in student populations and their corresponding relationship to achievement outcomes. Specifically, Mitchell and Mitchell (1999) noted that socioeconomic homogeneity within tracks occurs in situations where families are allowed to choose the track in which their child will enroll, and that differences in academic achievement between tracks are largely accounted for by these demographic differences. Unfortunately, the current study does not speak to the potential differences between single-track and multi-track SW year-round schools. No information was available in the extant database as to whether the 67 year-round schools were single or multi-track. Therefore, the school-wide year-round schools in this study likely include both types. However, a recent survey of year-round schools in North Carolina indicated that 87% are single-track (NCDPI, 2000), which would imply that the SW schools in the current study were largely single-track. Future studies of year-round schools would do well to address the possible differences between single and multi-track schools and whether either of those models has unique benefits for students, teachers or families. Future studies also need to differentiate between the effects of a year-round calendar and the effects of additional instructional time on student achievement. In North Carolina, almost all year-round programs offer some form of remediation and/or enrichment during intersessions, and 57% have mandatory intersession remediation for students who are behind academically (NCDPI, 1999b). This particular factor may be at least partially responsible for the slight benefits demonstrated for lower-achieving students in year-round programs in the current investigation. The question of whether the total amount of instructional time or the distribution of that time across the calendar year might be responsible for any achievement advantages for year-round schools has yet to be addressed in the research in this area. Further investigations which consider the length of time that a school has been year-round and which measure possible differences in pedagogical techniques between traditional and year-round schools are also needed. Studies such as these could help document factors that might mediate the effects of year-round education on student outcomes, and could determine whether being on a year-round schedule actually results in changes in day-to-day instructional activities.
Although increased achievement is often touted as a benefit of year-round
education, the results of the current investigation would suggest that the
merits of year-round education might best be judged on factors other than
achievement. Although a statistically significant interaction was found
indicating that lower-achieving students may benefit more from school-wide
year-round schools, this effect is probably educationally insignificant by most
standards. The consideration of other circumstances such as potential cost
savings and stakeholder preferences, which will vary from location to location, may provide a more reasonable basis
for decisions about whether to keep or adopt year-round calendars.
|