Does School Composition Matter? Estimating the Relationship between Race, Socioeconomic Status, and Achievement in Maryland Public Schools, 2015-2016

Jeremy R. Waldron, Robert Croninger, & Gail L. Sunderman 
October 2017

Download This Brief                                        

Summary of Key Findings

This analysis provides a look at the relationship between the racial/ethnic and economic composition of Maryland’s public schools and school performance. It shows that:

  1. There are differences between schools and school districts in the percentage of PARCC assessments on which students scored at or above proficient – what we refer to as a school’s proficiency rate.  The average school proficiency rate in some districts is as much as 40 percentage points lower than the average school proficiency rate in other districts. 
     
  2. The variation in PARCC school proficiency rates is highly correlated with the percentage of low- income students enrolled in a school. An increase in the percentage of low-income students in a school is associated with a decrease in a school proficiency rate on the PARCC assessments. The relationship is clearest at the state level, where the low-income student enrollment corresponds to nearly two thirds (R2=0.62) of the difference in proficiency rates between schools. 
     
  3. At the state level, the racial composition of a school also predicts PARCC performance, but the racial composition of a school is not as strong a predictor as the percentage of low-income students in a school.  An increase in black student enrollment is associated with a decrease in school proficiency rates, while an increase in white student enrollment is associated with an increase in school proficiency rates on the PARCC assessments.  At the state level the black student enrollment corresponds to nearly half (R2=0.43) and the white student enrollment corresponds to slightly more than one third (R2=0.35) of the difference in proficiency rates between schools. 
     
  4. While not all districts had a sufficient number of schools or variation in school composition to reliably estimate the relationship between school composition and school performance, the district analyses identified patterns of school composition and achievement similar to the state analysis.  Low-income enrollment in a school is the strongest predictor of a school’s proficiency rate, followed by black enrollment and white enrollment.  However, there is considerable variability across districts in how well a school’s low-income enrollment and racial/ethnic enrollment predict school proficiency rates on the PARCC assessments.

These findings are consistent with other research showing that school composition is related to student achievement.  However, these analyses do not provide information on how much of the association between school composition and achievement on the PARCC is due to differences in the quality of schooling or in the opportunities available to children outside of school. Identifying policies and practices to address these differences in school performance should be a priority for policymakers and education leaders throughout the state.

Introduction

Disparities in performance on standardized tests of academic achievement are well documented in education, especially differences in achievement between racial/ethnic and socioeconomic groups of students (Coleman et al., 1966; Jencks & Phillips, 1998; Lee, 2002, 2007).  Studies continue to document the persistence of achievement differences between racial/ethnic and socioeconomic groups of students nationwide, especially between children from high- and low-income families.  Over the past 50 years, as income inequality has risen, the achievement difference between socioeconomic groups of students has widened substantially to the point that the difference overshadows the achievement difference between racial/ethnic groups of students (Reardon, 2011, 2016).  What do achievement differences look like in Maryland?

One way to examine achievement differences is to examine differences in achievement between individual students.  In 2015, we examined data from the Maryland School Assessment (MSA) and the National Assessment of Education Progress (NAEP) from 2005 to 2013.  While the study found achievement gaps between minority (black and Hispanic students) and white students, and gaps between low-income students and economically advantaged students, the income achievement gaps were larger than the racial achievement gaps on the NAEP, a pattern consistent with national trends (Casalaspi, Sunderman, Croninger, & Luchner, 2015).  A second way to examine achievement differences, though, is to examine differences in achievement between schools.  Studies have found that going to a high-poverty or highly segregated school has a profound and negative impact on student outcomes (Borman & Dowling, 2010; Fantuzzo, LeBoeuf, & Rouse, 2014; Konstantopoulos & Borman, 2011; Mickelson, Bottis, & Lambert, 2013). Recent studies have found that the demographic composition of a school is more important than a student’s individual racial/ethnic or social class for understanding differences in educational outcomes (Borman & Dowling, 2010). 

In this policy brief, we examine differences between schools in achievement – specifically, differences between schools in the percentage of students who scored proficient or above proficiency on the 2015-2016 PARCC assessment.  We ask,

  1. Are there differences between school districts in school performance on the PARCC assessments? 
  2. Do schools that enroll higher percentages of low-income students report the same level of school performance on the PARCC assessments as schools that enroll lower percentages of low-income students? 
  3. Do schools that enroll higher percentages of black or white students report the same level of school performance on the PARCC assessments as schools that enroll lower percentages of black or white students? 
  4. What aspects of a school’s composition – percent low-income, percent black or percent white enrollment – have the strongest relationships with school performance on the PARCC assessments?

While these analyses are not causal – that is – they do not indicate that the racial/ethnic or socioeconomic composition of a school causes student achievement to increase or decrease, they do identify school-level patterns of achievement within the state worthy of attention.  Our analyses also provide an initial snapshot of patterns of achievement for schools using Maryland’s PARCC assessment, the central assessment instrument in the state’s newly revamped accountability system.  Because Maryland schools have become more racially, ethnically and economically diverse, as well as more segregated in the past twenty-five years (Sunderman & Dayhoff, 2014), there is a heightened need for policymakers, educators and other stakeholders to understand how academic performance, especially as measured by key accountability instruments, varies across schools attended by different populations of students.

Method

To estimate the relationship between school composition and school performance, we use 2015-2016 Maryland PARCC achievement scores downloaded from the Maryland State Department of Education (MSDE) school report card and 2014-2015 demographic data about school enrollments included in the Common Core of Data (CCD) downloaded from the National Center for Education Statistics (NCES).  To measure school performance, we calculated the percentage of PARCC examinations taken by students in a school on which students achieved a “Met Expectations” or “Exceeded Expectations” score, as indicated by a score of 4 or a 5, on the PARCC assessment.  We refer to this measure of school performance as a school’s success rate or proficiency rate.  Our proxy for income was the percentage of students in a school who qualify for free or reduced-priced meals (FARM).  Other composition variables included the percentage of black students enrolled in a school and the percentage of white students enrolled in a school.

 

To examine the relationship between school performance on the PARCC assessments and the demographic composition of schools we calculated a series of bivariate regressions, first for all schools in our statewide sample and then for schools within each school district that had a sufficient number of schools and variability in composition to estimate a model.  For our statewide analyses, we excluded schools with fewer than 100 students (18) and schools without matching MSDE and CCD IDs (120).  Our analytic sample for these analyses included 1,329 public schools in Maryland.  For our individual district analyses, we dropped school districts that had too few schools (fewer than 15) and districts with too little variability in a demographic variable (less than 15% or more than 85% of the average school enrollment for a district) to estimate reliably the regression models.  We dropped eight districts because they had fewer than 15 schools (Caroline, Dorchester, Garrett, Kent, Queen Anne’s, Somerset, Talbot and Worchester).  We dropped one district from the analyses for FARM enrollment because more than 85% of students were classified as FARM (Baltimore City); six districts from the analyses for black enrollment because less than 15% of the enrollment was black (Carroll, Washington, Frederick, Cecil, Calvert and Allegany), and four districts from the analyses for white enrollment because either the white enrollment was greater than 85% (Carroll and Allegany) or less than 15 percent (Baltimore City and Price George’s).  Our analytic sample that included FARM enrollment was 1080 schools in 15 districts, for black enrollment it was 1,040 schools in ten districts and for white enrollment it was 821 schools in 12 districts.

We report R-squared (R2) and the beta coefficient (bX) for each model.  R-squared provides an estimate of how well a demographic factor predicts a school’s success or proficiency rate on the PARCC assessment.  R-squared ranges between 0 and 1, where 0 predicts no relationship between school performance on the PARCC and school composition and 1 predicts a perfect relationship.  In general, the higher the R2, the better the model “fits” the data – or, in our case, the better a single demographic characteristic of a school’s enrollment predicts the academic performance of its students. The beta coefficient estimates the magnitude of the relationship between school performance and school enrollment, or the percentage point change in a school’s proficiency rate that corresponds to a percentage point change in school enrollment.  For the purpose of these analyses, we report the percentage point change in a school’s proficiency rate for every ten percentage point change in the demographic characteristics of a school’s enrollment (b=10X). 

While there is a relationship between R-squared and the beta coefficients in our regression models, each estimates a different aspect of the relationship between a school’s demographic composition and school performance.  When R-squared is relatively low, it suggests that other factors may be better predictors of school performance than the demographic variable considered in a model, even if the beta coefficient is statistically significant.  For each of the models that we report in this brief, R-squared and the beta coefficient provide complimentary but distinct estimates about the relationship between school achievement and enrollment composition – How well enrollment composition predicts school achievement (R2) and what is the difference in achievement between schools that vary by ten percentage points in enrollment composition (bX).  (See Appendix A for additional description of our methodology).

Description of Schools

Table 1 provides information about the number of schools, the average demographic composition of schools and the school proficiency rate on the PARCC assessments for the state and each district.  The first column reports the number of schools included in our state sample and for each of the school districts within the state.  The next three columns report the average school enrollment for each demographic factor; the last two columns report the average percentage of PARCC assessments on which students scored at proficiency or above and the difference between the state average percentage and the school district average percentage. 

As reported in Table 1, there is considerable variability between school districts in both the average demographic composition of schools and school performance on the PARCC assessments.  For the state, the average percentage of PARCC assessments on which students in a school scored proficient or above was 32.3%.  At the district level, the average percentage of PARCC assessments in a school on which students scored proficient or above ranged from a low of 10.4% in Baltimore City to a high of 51.6% in Worcester County.  Differences in the average school compositions for Baltimore City and Worchester County are also large, ranging from 37.6 percentage point difference in the enrollment of FARM students to a 60.1 percentage point difference in the enrollment of black students, providing preliminary evidence of a relationship between school achievement and composition across public schools in Maryland.  Schools in the district with the highest levels of FARM and black enrollments have the lowest levels of school achievement, while schools in the district with the lowest levels of FARM and black enrollment have the highest levels of achievement.

Table 1:         Number of Schools, Average School-Level Demographic Characteristics and Percent Proficient or Higher on the 2016 PARCC, State and District Samples

 

# of Schools

Average School
% FARM

Average School
% Black

Average School
% White

Average School
% Prof. or Higher

Minus State Avg. Prof.
or Higher

State Average

1329

48.6%

36.5%

40.2%

32.3%

NA

ALLEGANY

22

57.4%

2.7%

89.9%

33.1%

+1%

ANNE ARUNDEL

113

36.1%

20.5%

57.2%

40.3%

+8%

BALTIMORE CITY

162

85.2%

83.6%

8.1%

10.4%

-22%

BALTIMORE

172

49.2%

38.3%

42.5%

31.8%

-1%

CALVERT

22

24.0%

13.5%

72.2%

45.1%

+13%

CAROLINE

9

56.3%

16.7%

65.9%

33.0%

+1%

CARROLL

40

23.4%

3.9%

85.7%

47.5%

+15%

CECIL

27

45.1%

8.4%

78.6%

27.2%

-5%

CHARLES

36

39.9%

51.1%

31.9%

31.5%

-1%

DORCHESTER

11

63.5%

35.1%

50.6%

24.6%

-8%

FREDERICK

60

26.8%

10.4%

67.4%

42.1%

+10%

GARRETT

11

50.6%

0.2%

96.6%

30.8%

-2%

HARFORD

53

34.0%

18.1%

66.0%

41.3%

+9%

HOWARD

74

22.0%

23.1%

41.5%

47.8%

+15%

KENT

7

54.5%

21.7%

63.3%

29.5%

-3%

MONTGOMERY

189

35.9%

20.7%

31.6%

43.5%

+11%

PRINCE GEORGES

197

64.8%

65.3%

4.7%

19.7%

-13%

QUEEN ANNES

12

28.7%

7.0%

80.6%

40.0%

+8%

SOMERSET

7

83.1%

39.1%

47.1%

29.0%

-3%

ST. MARYS

25

34.8%

17.6%

66.1%

38.7%

+6%

TALBOT

8

41.7%

12.9%

69.7%

34.9%

+3%

WASHINGTON

41

46.9%

9.8%

75.7%

33.6%

+1%

WICOMICO

19

59.4%

33.0%

49.4%

29.7%

-3%

WORCESTER

12

47.6%

23.5%

62.5%

51.6%

+19%

Source: Maryland Report Card 2015-2016, NCES 2014-2015.

Maryland State Trends

Figures 1 through 3 provide a more detailed examination of the relationship between school achievement and each demographic factor for the state.  We present the results of our bivariate regression analyses for FARM student enrollment first, followed by the black student enrollment and white student enrollment.

Free and Reduced Price Meal Student Enrollment

Figure 1 plots the school proficiency rate on the PARCC assessments against the percentage of FARM student enrollment for each of the 1329 schools in our Maryland sample.  It shows that the percentage of FARM students in a school is a relatively good predictor of school performance on the PARCC (R2=0.62).  In the state model, the R-squared indicates that nearly two-thirds of the variability in school proficient rates is associated with a school’s FARM student enrollment.  Schools (each represented by a dot) are clustered close to the regression line, with PARCC scores generally decreasing as the percentage of FARM enrollment in a school increases.  For the state of Maryland, a ten percentage point increase in FARM enrollment corresponds to around a 5.2 percentage point decrease in a school’s proficiency rate on the PARCC assessments (b = -0.52 x 10).  

Figure 1:       Percent FARM Enrollment by School Proficiency Rate on PARCC Assessments
Figure 1
Source: Maryland Report Card 2015-2016, NCES 2014-2015.  *** Statistically significant at p < .001.

Black Student Enrollment

The percentage of black students enrolled in a school also predicts PARCC scores (R2=0.43), but not as well as the FARM student enrollment (R2=0.62).  Less than half of the variability in school proficiency rates is associated with a school’s black student enrollment.  As figure 2 shows, there is more variability in the relationship between black student enrollment and PARCC scores among schools, particularly among schools with low levels of black students.  The magnitude of the change in school proficiency rates is also not as large.  A ten percentage point increase in the proportion of students in a school that is black is associated with a 3.8 percentage point decrease in a school’s proficiency rate on the PARCC assessments (b = 0.38 x 10).  

Figure 2:          Percent Black Enrollment by School Proficiency Rate on PARCC Assessments
Figure 2
Source: Maryland Report Card 2015-2016, NCES 2014-2015.  *** Statistically significant at p < .001.

White Student Enrollment

Finally, the percentage of students in a school that are white is the weakest predictor of school proficiency rates (R2= 0.35).  Roughly one-third of the variability in achievement among schools is associated with white student enrollment.  As shown in figure 3, there is considerable variability of schools around the regression line; the spread among the dots representing schools is substantially greater than in either of the previous figures.  In this case, rather than a decrease in school proficiency rates, a ten percentage point increase in the proportion of white students is associated with a 3.4 percentage point increase in a school’s proficiency rate on the PARCC assessments (b = 0.34 x 10).

Figure 3:     Percent White Enrollment by School Proficiency Rate on PARCC Assessments
Figure 3
Source: Maryland Report Card 2015-2016, NCES 2014-2015.  *** Statistically significant at p < .001.

District Level Comparisons

While the state-level analysis suggests that school-level demographic composition is related to school performance on the PARCC, these relationships are likely to vary by district.  School performance levels and student demographics are not evenly distributed across districts—some districts have higher average school performance levels than others, higher poverty rates than others, and some enroll more minority students (see Table 1).  The district analysis will tell us to what extent the state patterns are repeated at the district level.

Table 2 identifies the school districts included in the district-level analyses.  Column one reports the number of schools included in the statewide analysis for each school district.  The next three columns report whether a school district was included in the analysis for each of the demographic factors.  Recall that we did not include school districts in the analysis if they had fewer than 15 schools (S), or if the average school enrollment for a factor was less than 15% or greater than 85% (D).  We included districts that satisfied both criteria in our district-level analysis (I).  See Table 1 for the average demographic enrollment for districts excluded and included in Table 2.  As with the statewide analyses, we report the regression results for FARM student enrollment first, next the regression results for black student enrollment and last the regression results for white student enrollment.

Table 2:         School Districts Included in the District-Level Analyses

District

# of Schools

Regression Models

FARM

Black

White

HOWARD

74

I

I

I

MONTGOMERY

189

I

I

I

CARROLL

40

I

D

D

BALTIMORE

162

I

I

I

ST. MARY’S

25

I

I

I

HARFORD

53

I

I

I

WASHINGTON

41

I

D

I

ANNE ARUNDEL

113

I

I

I

CHARLES

36

I

I

I

FREDERICK

60

I

D

I

WICOMICO

19

I

I

I

CECIL

27

I

D

I

CALVERT

22

I

D

I

PRINCE GEORGE’S

197

I

I

D

ALLEGANY

22

I

D

D

BALTIMORE CITY

172

D

I

D

CAROLINE

9

S

S

S

DORCHESTER

11

S

S

S

GARRETT

11

S

S

S

KENT

7

S

S

S

QUEEN ANNE’S

12

S

S

S

SOMERSET

7

S

S

S

TALBOT

8

S

S

S

WORCESTER

12

S

S

S

Source: Maryland Report Card 2015-2016, NCES 2014-2015.
Note: I = included in the analysis.  D = average district level demographic composition is less than 15% or more than 85% for any subgroup. S = fewer than 15 schools in a district. 

Free and Reduced Priced Meal Student Enrollment

Table 3 shows the R-squared value at the district level as well as an estimate of the percentage point change in school performance for a ten percentage point increase in the FARM representation.  Figure 4 maps the strength of the relationship (R2), while Figure 5 maps the magnitude of the relationship (b).  Each figure is color coded with “warmer” colors representing a higher R-squared value or higher beta coefficient.  With the exception of Allegany County, there is a statistically significant relationship between FARM enrollment and school performance in every district included in our analyses, though the strength and magnitude of the relationship varies by district.  The strength of the relationship (R2) ranges from a high of 0.73 to a low of 0.09, whereas the magnitude of the relationship (b) ranges from a high of -7.58 to a low of -1.75.

The relationship between FARM enrollment and PARCC is strongest in Howard County (R2=0.73), where a ten percentage point increase in FARM enrollment is associated with a 7.58 percentage point decrease in school proficiency on the PARCC assessments.  By comparison, FARM enrollment is a weaker predictor of PARCC proficiency in Prince George’s County (R2=0.13), where a ten percentage point increase in FARM enrollment is related to a 1.75 percentage point decrease in average school performance; in Allegany County the relationship between FARM enrollment and PARCC proficiency is not statistically significant.  

Compared to the state model, Howard County is the only district model where there is a stronger relationship between FARM student enrollment and school performance (R2).  Nonetheless, FARM enrollment is associated with at least one-quarter of the variability in school performance for almost all of the district models, Prince George’s and Allegany counties being the exceptions.  The magnitude of the relationship between FARM enrollment and a decrease in performance is greater in five districts than in the state model.  In addition to previously mentioned Howard County, Calvert (-6.60), Baltimore (-5.68), Montgomery (-5.51) and Carroll (-5.42) all have larger beta coefficients than reported for the state model.  Two districts – Harford County (-4.37) and Anne Arundel County (-4.21) – are within one percentage point of the state’s estimated coefficient.

Table 3:         Percent FARM Enrollment by School Proficiency Rate on PARCC Assessments by District

District

# of Schools

Regression Results

R-squared

Beta Coefficient

State Analysis

1329

0.62

              -5.21***

HOWARD

74

0.73

              -7.58***

MONTGOMERY

189

0.59

              -5.51***

CARROLL

40

0.56

               -5.42***

BALTIMORE

162

0.54

               -5.68***

ST. MARY’S

25

0.49

               -3.63***

HARFORD

53

0.45

               -4.37***

WASHINGTON

41

0.39

               -3.28***

ANNE ARUNDEL

113

0.38

               -4.21***

CHARLES

36

0.33

               -3.81***

FREDERICK

60

0.30

               -3.64***

WICOMICO

19

0.26

               -2.55*

CECIL

27

0.25

               -2.89**

CALVERT

22

0.25

               -6.60*

PRINCE GEORGE’S

197

0.13

               -1.75***

ALLEGANY

22

0.09

               -2.80

BALTIMORE CITY

172

D

D

CAROLINE

9

S

S

DORCHESTER

11

S

S

GARRETT

11

S

S

KENT

7

S

S

QUEEN ANNE’S

12

S

S

SOMERSET

7

S

S

TALBOT

8

S

S

WORCESTER

12

S

S

Source: Maryland Report Card 2015-2016, NCES 2014-2015.
Note: D = average school level FARM enrollment for the district is more than 85%.  S = fewer than 15 schools in a district.  *** Statistically significant at p < .001; ** at p < .01 and * at p < .05.

Figure 4:       Strength of Relationship between Percent FARM Enrollment and School Proficiency Rate on PARCC Assessments by District (R-Squared)
Figure 4
Source: Maryland Report Card 2015-2016, NCES 2014-2015.
Figure 5:       Magnitude of Relationship between Percent FARM Enrollment and School Proficiency Rate on PARCC Assessments by District (Beta Coefficient)
Figure 5
Source: Maryland Report Card 2015-2016, NCES 2014-2015.

Black Student Enrollment

Table 4 presents the R-squared value at the district level as well as an estimate of the percentage point change in school performance for a ten percentage point increase in the black student enrollment.  Figure 6 maps the strength of the relationships (R2), while Figure 7 maps the magnitude of the relationships (b).  Each figure is color coded with “warmer” colors representing a higher R-squared value or higher beta coefficient.  There is a statistically significant relationship between black student enrollment and school performance in every district, except Prince George’s County.  But, as with the analysis of FARM student enrollment, the strength and magnitude of the relationship varies by district.  The strength of the relationship (R2) ranges from a high of 0.71 to a low of 0.00, whereas the magnitude of the relationship (b) ranges from a high of -8.29 to a low of -0.11.

Among the ten districts that met our criteria for demographic diversity, the percentage of black students in a school was the strongest predictor of PARCC outcomes in Howard County (R2=0.71), where a 10 percentage point increase in black students attending a school is associated with a 8.29 percentage point decrease in a school’s proficiency rate on the PARCC assessments.  Black student enrollment is a far less reliable predictor of school performance in Baltimore City (R2=0.16) and Prince George’s County (R2=0.00).  In these districts, a ten percentage point increase in black student enrollment is associated with a -1.90 and -0.11 decrease in a school’s proficiency rate, respectively.

 Compared to the state model, the models for Howard County and Harford County (R2=0.46) report a stronger relationship between black student enrollment and school performance, while the model for Charles County (R2=0.42) is nearly identical to the state model.  However, similar to the analysis of FARM student enrollment, black student enrollment is associated with at least one-quarter of the variability in school performance for most of the district models, Baltimore City and Prince George’s County being the exceptions. In four districts models, including Howard County, the magnitude of the relationship between FARM enrollment and a decrease in performance is greater than in the state model: Montgomery (-6.58), Harford (-5.89) and Anne Arundel (-4.97) counties.  The magnitude of the relationship in Charles (-3.77), Baltimore (-3.52), St. Mary’s (-3.16) and Wicomico (-2.90) counties are within one percentage point of the state’s estimated coefficient.

Table 4:         Relationship of Black Student Enrollment and School Proficiency Rates on PARCC Assessments by District

District

# of Schools

Regression Results

R-Squared

Beta Coefficient

State Analysis

1329

0.43

                -3.80***

HOWARD

74

0.71

                -8.29***

HARFORD

53

0.46

                -5.89***

CHARLES

36

0.42

                -3.77***

WICOMICO

19

0.39

                -2.90**

ANNE ARUNDEL

113

0.29

                -4.97***

ST. MARY’S

25

0.29

                -3.16*

BALTIMORE

162

0.28

                -3.52***

MONTGOMERY

189

0.28

                -6.58***

BALTIMORE CITY

172

0.16

                -1.90***

PRINCE GEORGE’S

197

0.00

                -0.11

ALLEGANY

22

D

D

CALVERT

22

D

D

CARROLL

40

D

D

CECIL

27

D

D

FREDERICK

60

D

D

WASHINGTON

41

D

D

CAROLINE

9

S

S

DORCHESTER

11

S

S

GARRETT

11

S

S

KENT

7

S

S

QUEEN ANNE’S

12

S

S

SOMERSET

7

S

S

TALBOT

8

S

S

WORCESTER

12

S

S

Source: Maryland Report Card 2015-2016, NCES 2014-2015.
Note: D = average school level black student enrollment in the district is less than 15%.  S = fewer than 15 schools in a district.  *** Statistically significant at p < .001; ** at p < .01 and * at p < .05.

Figure 6:       Strength of Relationship Between Percent Black Enrollment and School Proficiency Rates on PARCC Assessments by District (R-Squared)
Figure 6
Source: Maryland Report Card 2015-2016, NCES 2014-2015.
Figure 7:     Magnitude of Relationship Between Percent Black Enrollment and Proficiency Rates on PARCC Assessments by District (Beta Coefficient)
Figure 7
Source: Maryland Report Card 2015-2016, NCES 2014-2015.

White Student Enrollment

Table 5 displays the R-squared values and the change in school performance associated with a ten percentage point increase in white student enrollment for the districts that qualified for these analyses.  Figure 8 maps the strength of the relationships (R2), while Figure 9 maps the magnitude of the relationships (b).  Each figure is color coded with “warmer” colors representing a higher R-squared value or higher beta coefficient.  There is a statistically significant relationship between the white student enrollment and school performance in every district except Calvert County.  As with the other analyses, the strength and magnitude of the relationship varies by district.  The strength of the relationship (R2) ranges from a high of 0.44 to a low of 0.08, whereas the magnitude of the relationship (b) ranges from a high of 6.33 to a low of 1.58.

White student enrollment was the strongest predictor of PARCC scores in Howard (R2=0.44), with a ten percentage point increase in white students related to a 6.33 percentage point increase in a school’s proficiency rate on the PARCC assessments.  By comparison, white student enrollment was the weakest predictor of school performance in Washington (R2=0.10) and Frederick (R2=0.08) counties.  In these districts, a ten percentage point increase in the white student enrollment corresponds to a 1.95 and a 1.58 percentage point increase in school-level achievement, respectively.  As mentioned previously, the relationship was not statistically significant in Calvert County.

When compared to the state model, white student enrollment is a stronger predictor of school performance in Howard, Montgomery (R2=0.42) and Harford counties (R2=0.41), while the models for Cecil County (R2=0.35) and Anne Arundel County (R2=0.34) are identical or nearly identical to the state model.  The relationship between white student enrollment and PARCC performance in three other districts corresponds to at least one-quarter of the variability in performance among schools in those counties (Charles, Wicomico and Baltimore).  Five district models, including Howard County, report a greater magnitude for the relationship compared to the state model:  Montgomery (5.37), Harford (4.41), Anne Arundel 3.83) and Baltimore (3.50) counties.  While the estimate for Calvert County is also greater (5.72) than the state estimate (3.34), it is not statistically significant.  Cecil (3.23), and Charles (3.01) counties are within one percentage point of the state’s estimated coefficient.

Table 5:         Percent White Enrollment by School Proficiency Rates on PARCC Assessments by District

District

# of Schools

Regression Results

R-Squared

Beta Coefficient

State Analysis

1329

0.35

               3.34***

HOWARD

74

0.44

               6.33***

MONTGOMERY

189

0.42

               5.37***

HARFORD

53

0.41

               4.41***

CECIL

27

0.35

               3.23***

ANNE ARUNDEL

113

0.34

               3.83***

CHARLES

36

0.32

               3.01***

WICOMICO

19

0.30

               2.16*

BALTIMORE

162

0.26

               3.50***

ST. MARYS

25

0.21

               2.11*

CALVERT

22

0.10

               5.72

WASHINGTON

41

0.10

               1.95*

FREDERICK

60

0.08

               1.58*

ALLEGANY

22

D

D

BALTIMORE CITY

172

D

D

CARROLL

40

D

D

GARRETT

11

D

D

PRINCE GEORGES

197

D

D

CAROLINE

9

S

S

DORCHESTER

11

S

S

KENT

7

S

S

QUEEN ANNES

12

S

S

SOMERSET

7

S

S

TALBOT

8

S

S

WORCESTER

12

S

S

Source: Maryland Report Card 2015-2016, NCES 2014-2015.
Note: D = average school-level White enrollment in the district is less than 15% in Baltimore City and Prince George’s County and more than 85% in Allegany, Carroll, and Garrett counties. S = fewer than 15 schools in a district.  *** Statistically significant at p < .001; ** at p < .01 and * at p < .05.

Figure 8:       Strength of Relationship Between Percent White Enrollment and School Proficiency Rates on PARCC Assessments by District (R-Squared)
Figure 8
Source: Maryland Report Card 2015-2016, NCES 2014-2015. Figure 9:       Magnitude of Relationship Between Percent White Enrollment and School Proficiency Rates on PARCC Assessments by District
Figure 9
Source: Maryland Report Card 2015-2016, NCES 2014-2015.

Conclusions and Implications

These analyses indicate that there is a clear and relatively persistent relationship between the percentage of PARCC assessments on which students score at or above proficiency in a school and the school’s demographic composition, especially the percentage of students in a school who qualify for free or reduced-price meals.  The relationship is clearest at the state level, where the FARM student enrollment corresponds to nearly two thirds (R2=0.62), the black student enrollment corresponds to nearly half (R2=0.43) and the white student enrollment corresponds to slightly more than one third (R2=0.35) of the difference in the proficiency rates between schools. 

Moreover, the magnitude of the relationship between school performance and school composition is not trivial.  For example, the beta coefficient for FARM student enrollment is -0.521 (see Table 3).  Let’s say one school has 90% FARM enrollment and another school has the average, 48.6%, FARM enrollment (see Table 1).  The difference between the schools in FARM student enrollment is 41.4 percentage points (90% - 48.6% = 41.4 percentage points).  Therefore, the difference in school proficiency between the school with 90% FARM enrollment and the school with average FARM enrollment is roughly -22 percentage points (41.4 x -.521 = -21.5).  Given that the average school proficiency rate is 32.3%, this is a large and disturbing difference.  Estimated difference in school performance between a school with average black enrollment (36.5%) and 90% enrollment is -20 percentage points; the difference in school performance between a school with average white enrollment (40.2%) and 90% enrollment is 17 percentage points.

While the results of our analyses were more mixed when examined at the district level, the patterns were relatively consistent with what we found with the state analysis.  For most of the counties that qualified for the analyses, there was a statistically significant relationship between the school’s composition and school performance on the PARC assessments.  Exceptions were Allegany County (FARM student enrollment); Prince George’s County (black student enrollment) and Calvert County (white student enrollment).  Nonetheless, we found statistically significant relationships between school composition and school performance in 34 of the 37 district-level regression models that we reported for this brief.  

Similar to the state analysis, FARM student enrollment was the strongest predictor of school performance at the district level, followed by black student enrollment and white student enrollment.  The average R-squared for the district models was 0.38 for FARM student enrollment, 0.33 for black student enrollment and 0.28 for white student enrollment; the average beta coefficient for the district models was -4.25 for FARM student enrollment, -4.11 for black student enrollment and 3.60 for white student enrollment.  Although the average R-squared and beta coefficients for the district analyses are smaller than the values for the state analysis (with the exception of the beta coefficient for white enrollment), the pattern of results is roughly the same  –  the largest gap in school performance is between schools attended by higher and lower income students, followed students who attend schools with higher and lower percentages of black and white enrollments.    

These findings are consistent with other research showing that school composition is related to student achievement.  However, these analyses do not provide information on how much of the association between school composition and achievement on the PARCC is due to differences in the quality of schooling or in the opportunities available to children outside of school.  Many factors shape test scores and academic performance, including a child’s family resources, their home environment, the neighborhood where a child lives, their pre-school and after-school experiences, as well as what happens in school itself.  Test scores do not tell us whether one school is better than another and are best interpreted as a measure of the educational opportunities available to children, and how well these opportunities are distributed across schools in a particular district or state.

Recognizing the school compositional sources of achievement disparities can direct Maryland policymakers’ attention to policies that address the challenges facing schools that serve diverse populations of students.  This is particularly important in Maryland where the public schools are becoming more economically and racially diverse, and at the same time more segregated by both race and income (Sunderman & Dayhoff, 2014).  As schools have become more diverse, the concentration of low-income students in schools has increased substantially – in 2014 almost half (46.9%) of all schools in Maryland enrolled 50% or more low-income students, compared to 14.2% in 1990.  The percentage of racially isolated minority schools (i.e., schools that are 90-100% minority) also increased from 12.4% of all schools in 1990 to 28.4% in 2014.

What are the implications of these findings for policy?  The especially strong relationship between the percentage of low-income students in a school and school performance suggests a need to adopt research-based policies that mitigate the effects of poverty on student outcomes and reduce the achievement gap between economically disadvantaged and more advantaged students.  This could include policies that encourage racial and socioeconomic integration both in schools and in neighborhoods.  Research has shown that policies that promote greater integration, including policies that influence residential segregation, can improve achievement and reduce differences in achievement between schools (Schwartz, 2013). 

We also encourage policies that expand access to prekindergarten, summer school and after school programs, and support for community school models that connect students and their families to local services. Kindergarten programs enhance children’s readiness for schools, improve their social skills, and have benefits that extend across a student’s academic career and shape their later life outcomes (Sunderman & Titan, 2014).  Research suggests that it is among the most cost-effective educational inventions a society can make (Garcia, Heckman, Leaf, & Prados, 2017).  Similarly, summer school has been shown to address the summer learning loss among low-income students (Borman & Dowling, 2006), and after school programs that are connected to a student’s curricular program enhance student learning (Joseph, Weissberg & Molly, 2010).  Community school models seek to expand the traditional educational mission of the school to include health and social services for children and families with the goal of improving the overall wellbeing of children (Valli, Stefanski, & Jacobson, 2014).  Community schools integrate student support services at the school site and focus on non-academic factors such as health, nutrition, and other aspects of a child’s development.  

Finally, we encourage greater attention to interventions that develop non-cognitive and social skills since these play a role in achieving successful lives, too (Cunha & Heckman, 2010).  A number of intervention studies show that ability gaps, including social skills gaps, can be reduced through early remediation (Cunha & Heckman, 2010).  Adopting these and similar proactive policies and programs may enhance Maryland’s ability to address the challenges facing the state’s schools in achieving proficiency on the PARCC assessments for all students, regardless of race/ethnicity or family income.

 

Appendix A: Methodology and Technical Information

All analyses for this paper used 2016 Maryland PARCC scores, found at Maryland Report Card’s data download website.  We matched those data to 2014-2015 enrollment data collected by the NCES Common Core of Data.  When there was no match, or when schools had fewer than 100 students, we excluded those schools from our analyses.  A list of excluded schools can be found in Appendix B.

The Maryland Report Card provides the number of students that achieve a 4 or a 5 (proficient or above) on each PARCC assessment (English/Language Arts and Mathematics).  We added the number of students in a school reported to have scored a 4 or 5 on each assessment, and then divided that number by the total number of assessments taken by students in a school.  Where data in the report was suppressed due to a small cell size (too few students who scored at a particular level), we estimated the number of students who achieved a specific PARCC score using the total number of students who took an assessment.  The resulting measure equals the percent of PARCC assessments taken by students in a school in which students achieved a 4 or 5.  We refer to this measure as a school’s success or proficiency rate. 

For the state analysis, we modeled school performance – the school’s success or proficiency rate – using the percentage of FARM, black or white students enrolled in that school.  We used simple, bivariate linear regression to do so.  We reported R2, unadjusted for sample size, as a measure of the strength of a relationship and the unstandardized beta coefficient for school composition as a measure of the magnitude of a relationship.  We multiplied the regression coefficient by ten so that the beta coefficient equaled a change in the success or proficiency rate on the PARCC assessment for every ten percentage point change in a school’s composition.  In the district analyses, we dropped districts that had too few schools or too little variability in a demographic factor to estimate reliably a regression model. 

Limitations

This study is a correlational analysis and does not imply causality.  We cannot say, for example, that the composition of a school causes higher or lower levels of school performance.  Rather, other factors, related to school composition, undoubtedly influence school performance, as we noted in the section “Conclusions and Implications”.  Our analysis provides a preliminary baseline for understanding the relationship between school composition and school performance.  Identifying the causes of that relationship will require additional data and more complex, sophisticated analyses.

Our measure of school performance is not the same as the percentage of students who achieved proficiency or above on one or more of the PARCC assessments.  While such an analysis would be informative, especially if it included an analysis of differences in student populations that did so, we did not have access to data that would permit a more fine-grained analysis of school performance.  Instead, we focus on the relationship between school composition and the percentage of PARCC assessments on which students achieved proficiency or above in a school.  We argue that a school’s proficiency rate, as measured by student’s success across all the PARCC assessments that they took in a particular year, is a good, though recognizably incomplete, indicator of school performance.

We assumed that the relationship between school composition and school performance was linear – that is, we did not examine whether there might be “tipping points” associated with school composition.  For example, it is possible that the relationship between school enrollment and school performance changes when a school has either a relatively low or relatively high concentration of particular student populations.  We report the average increase or decrease in school performance for a ten percentage point change in school composition.  More precise estimates of change would require exploring the possibility of a non-linear relationship between school composition and school performance.

Our analysis focuses on the relationship between school composition and school performance.  We did not examine the relationship between an individual student’s race/ethnicity or family income and a student’s performance on the PARCC assessment.  Generally speaking, it can be misleading to infer statistical relationships from one unit of analysis (e.g., schools) to another unit of analysis (e.g., individuals), because these relationships often differ, sometimes substantially, from each other (Raudenbush & Bryk, 2002).  For example, because we could not estimate the relationship between school composition and performance for a number of districts, it does not mean that there is no relationship between individual student race/ethnicity or family background and individual achievement.  Regardless of the district, estimating the relationship between student’s race/ethnicity, family income and performance on PARCC requires different data and a different form of analysis than what we report here. 

Appendix B: Omitted Schools:  See pdf.

References

Borman, G. D., & Dowling, N. M. (2006). Longitudinal achievement effects of multiyear summer school: evidence from the Teach Baltimore randomized field trial. Educational Evaluation and Policy Analysis, 28(1), 25-48.

Borman, G. D., & Dowling, N. M. (2010). Schools and inequality: A multilevel analysis of Coleman's equality and educational opportunity data. Teachers College Record, 112(5), 1-2. Retrieved from http://www.tcrecord.org

Casalaspi, D., Sunderman, G. L., Croninger, R. G., & Luchner, J. (2015). Making sense of MSA and NAEP assessments results: How well are Maryland students doing? Retrieved from College Park, MD: http://www.education.umd.edu/TLPL/centers/MEP/Research/k12Education/MEP_MD Student Achievement_MSA-NAEP_9 15 15.pdf

Coleman, J. S., Campbell, E. Q., Hobson, C. J., McPartland, J., Mood, A. M., Weinfield, F. D., & York, R. L. (1966). Equality of educational opportunity. Retrieved from Washington, DC:

Cunha, F., & Heckman, J. J. (2010). Investing in our young people. Retrieved from Bonn, Germany: https://www.econstor.eu/bitstream/10419/44199/1/631911804.pdf

Fantuzzo, J. W., LeBoeuf, W. A., & Rouse, H. L. (2014). An investigation of the relations between school concentrations of student risk factors and student educational well-being. Educational Researcher, 43(1), 25-36.

Garcia, J. L., Heckman, J. J., Leaf, D. E., & Prados, M. J. (2017). Quantifying the life-cycle benefits of a prototypical early childhood program. Retrieved from Cambridge, MA:

Jencks, C., & Phillips, M. (1998). The black-white test score gap. Washington, DC: Brookings Institution Press.

Joseph, D. A., Weissberg, R. P., & Molly, P. (2010). A meta-analysis of after-school programs that seek to promote personal and social skills in children and adolescents. American Journal of Commmunity Psychology, 45(3-4), 215-453.

Konstantopoulos, S., & Borman, G. D. (2011). Family background and school effects on student achievement: A multilevel analysis of the Coleman data. Teachers College Record, 113(1), 97-132.

Lee, J. (2002). Racial and ethnic achievement gap trends:  Reversing the progress toward equity? Educational Researcher, 31, 3-12.

Lee, J. (2007). The testing gap: Scientific trials of test-driven school accountability systems for excellence and equity. Charlotte, NC: Information Age Publishing.

Mickelson, R. A., Bottis, M. C., & Lambert, R. (2013). Effects of school racial composition on K-12 mathematics outcomes: A metagregression analysis. Review of Educational Research, 83(1), 121-158.

Raudenbush, S. W., & Bryk, A. S. (2002). Hierarchical linear models. Applications and data analysis methods (2nd edition). Thousand Oaks, CA: Sage

Reardon, S. F. (2011). The widening academic achievement gap between the rich and the poor: New evidence and possible explanations. In G. J. Duncan & R. J. Murnane (Eds.), Whither opportunity? Rising inequality, schools, and children's life chances (pp. 91-115). New York: Russell Sage Foundation.

Reardon, S. F. (2016). School district socioeconomic status, race, and academic achievement. Retrieved from Palo Alto, CA: https://cepa.stanford.edu/sites/default/files/reardon district ses and achievement discussion draft april2016.pdf

Schwartz, H. (2013). The potential of economic integration to raise academic achievement for low-income students. In G. L. Sunderman (Ed.), Charting reform, achieving equity in a diverse nation. Charlotte, NC: Information Age Publishing, Inc.

Sunderman, G. L., & Dayhoff, J. (2014). Creating opportunities or settling for inequities? Two decades of change in Maryland's public schools. Retrieved from College Park, MD: http://www.education.umd.edu/TLPL/centers/MEP/Research/inequities/MEP_DemographicReport_2014.pdf

Sunderman, G. L., & Titan, C. (2014). Can Maryland benefit from universal preschool? A review of the research on the efficacy of early education. Retrieved from College Park, MD:

Valli, L., Stefanski, A., & Jacobson, R. (2014). School-community partnerships: A typology for guiding systemic educational reform. Retrieved from College Park, MD: http://www.education.umd.edu/TLPL/centers/MEP/Research/k12Education/Scho...

 

About the Authors

Jeremy R. Waldron, B.S., is research assistant with the Maryland Equity Project. He is a master's of public policy candidate in the field of international economics, security, and development as well as a MBA candidate with the Smith School of Business, University of Maryland.

Robert Croninger, Ph.D., is Co-Director of the Maryland Equity Project, associate professor in the Department of Teaching and Learning, Policy and Leadership in the College of Education at the University of Maryland. 

Gail L. Sunderman, Ph.D., is Co-Director of the Maryland Equity Project and senior research scientist in the Department of Teaching and Learning, Policy and Leadership in the College of Education at the University of Maryland.

Corresponding authors:  Robert Croninger or Gail L. Sunderman, The Maryland Equity Project, College of Education, University of Maryland.