May 2021: Child Trend’s Final Report entitled, “Understanding Equitable Access to Public Montessori pre-K: A Case Study of Montessori Recruitment and Enrollment Practices,” is released. The Executive Summary can be found here. The entire Final Report can be found here. Child Trends’ blog post discussing the Final Report can be found here.)
The BEF Montessori Initiative Network: Changing Directions to Support Equitable Learning Opportunities
September 2019: The Montessori method is a whole-child approach that focuses on developing the intellectual, moral, emotional and behavioral skills of children. Recently, BEF embarked on an initiative that aimed to investigate the impact of Primary Montessori Education (PreK-K) in the public sector, specifically focusing on if the approach is equally effective across cultural groups and whether it moderates the effects of family income on children’s academic achievement, executive functioning, and social outcomes.
An Advisory Board was formed to guide the initiative, and three research teams were selected to create the Brady Education Foundation Montessori Initiative Network (BEFMIN). Teams from Child Trends and the Riley Institute at Furman University were selected to collect data, and a team from the University of Kansas Center for Montessori Research was selected to serve as the coordinating site for the initiative. A number of schools also agreed to participate in the study, and BEF is very grateful to the schools and school systems that generously agreed to partner with the network to investigate these critical questions.
Recently, the BEF Board arrived at the difficult conclusion that the original study is not feasible due to a variety of challenges the teams faced attempting to conduct the study. (please see below for more detail concerning these challenges)
However, important lessons were learned attempting to conduct the study, and the network has transformed its work in response to these challenges to meet the overarching aim of the initiative of informing philanthropic and public policy efforts to create equitable learning opportunities for all children.
In this new direction, rather than working on one multi-year study, the teams will work as a true network conducting related, but separate, projects that contribute to the underlying goals of the initiative:
Child Trends, in collaboration with Furman University, will conduct a public policy/landscape study to investigate the extent to which access to public Montessori pre-K programs is equitable.
The Riley Institute at Furman University will conduct a quasi-experimental study using administrative data to inform whether Montessori education in the public sector is equally effective across racial groups and whether it reduces the association between family income and children’s outcomes in pre-k/K and older grades.
The University of Kansas Center for Montessori Research will engage in two activities that have the potential to contribute to future work in this area: continuing their measurement development work and holding their next annual Montessori Research Retreat with an enhanced focus on equity.
The following provides greater detail concerning the the creation of this network, the challenges faced conducting the study, and how the network has transformed its work in response to these challenges to meet the overarching aim of the initiative of informing philanthropic and public policy efforts to create equitable learning opportunities for all children.
The Opportunity and Achievement Gaps
The policies and practices that form the public education system in the U.S. create and maintain inequality in educational outcomes for children of color and children raised in families with low economic resources compared to their White and more economically advantaged peers, as evidenced in the numerous empirical studies documenting the achievement gaps.1 Scholars have noted the importance of adjusting the focus of this work from “achievement gaps” to opportunity gaps, shifting the attention from outputs (i.e., children’s outcomes) to inputs (i.e., the inequities in society and educational systems that result in reduced access to high-quality educational settings and other resources).2
A wide variety of factors have been associated with these educational disparities including both structural factors (such as inequitable distribution of resources for schools) and interpersonal ones (such as explicit and implicit biases that contribute to reduced academic expectations and discriminatory discipline practices).3 That’s the bad news. The good news is that educators, researchers, philanthropists, and policymakers are increasingly focused on these disparities and are more motivated than ever to address them, as evidenced by the numerous initiatives focused on disrupting institutional racism, inequities, and bias and creating equitable learning opportunities.
The BEF’s mission is to close the educational opportunity gaps associated with race and family income, thus focusing on these disparities is central to our work. BEF endeavors to fulfill its mission via the funding of research and program evaluations that inform both private philanthropy (such as the Bezos Day One Fund) and public policy (given the need to address the current systems that create and maintain structural inequality). With this mission in mind, the Foundation embarked on an initiative intended to support a network of researchers to conduct a rigorous study of the potential of Montessori early education in the public sector to address the racial and economic disparities in educational achievement and success.
Creating the BEFMIN
In 2009, BEF funded a longitudinal study that investigated the impact of Montessori education at the primary level (PreK3 – K) in two public magnet schools in Hartford, CT. The results, released in 2017, demonstrated positive impacts for Montessori early education overall and also suggested that the approach may help to close opportunity gaps in the public sector.4 Soon after these results were published, the findings from a large study of Montessori education in South Carolina public schools were released, with results that also suggested that Montessori has a positive impact overall and may help to close the racial and economic gaps.5
While the findings from these studies were impressive, as with all studies, each had limitations. A strength of the Hartford study was that it capitalized on lotteries to conduct a randomized control trial (RCT), the gold standard in research to investigate effects. However, it was conducted in only two schools in one geographic location, it lacked the sample size and diversity needed to examine whether children of color equally benefited from Montessori, and it could not compare income groups based upon policy relevant income categories (e.g., students who qualified for free and reduced price lunch). In contrast, the South Carolina study did have a large sample that was ethnically and economically diverse. However, their quasi-experimental design limited the ability to conclusively confirm that Montessori was indeed effective in reducing the economic and racial disparities.
Despite these limitations, the results of these and other studies were compelling enough to lead the BEF Board to embark on a Montessori initiative. BEF issued a Request for Proposals (RFP) to conduct a large and rigorous multi-site study to investigate the impact of primary Montessori education (PreK3-K) in the public sector on children’s academic achievement, executive functioning, and social outcomes, specifically focusing on:
If the approach is equally effective across cultural groups that make up the majority of families in the U.S. (Black/African Heritage, Latine6 and Hispanic Heritage, and White/European Heritage);
Whether it is an effective means for reducing the effects of family income on academic outcomes.
The goals of the BEF initiative had important implications for the criteria used to select the research teams that responded to the RFP. Given the focus on diverse populations, the BEF Board determined that the research teams conducting the work and the Advisory Board guiding the initiative needed to not only have solid Montessori expertise and strong research experience, but also reflect the populations being studied at the leadership level. For too long, the vast amount of research conducted with children of color has been designed and conducted primarily by White researchers, which likely limits the types of questions asked, the information gathered, and the interpretations of the findings.
An Advisory Board was formed to guide the initiative and three research teams were selected to create the Brady Education Foundation Montessori Initiative Network (BEFMIN). Teams from Child Trends and the Riley Institute at Furman University were selected to collect data, and a team from the University of Kansas Center for Montessori Research was selected to serve as the coordinating site for the initiative. Once the network was created, the teams went to work seeking schools and school systems to partner with us, developing data collection protocols, and recruiting families to participate to prepare for the first wave of data collection in the fall of 2019.
The goal of generating valid findings that can best inform private investment and public policy requires a strong research design. Three methodological issues were identified as particularly important early on given the primary aims of this study.
In order to have the ability to conclusively attribute child outcomes to the Montessori approach rather than other factors (such as family characteristics that lead parents to select one type of program over another), an RCT would need to be conducted (such as when schools with more applicants than placements available use lotteries to randomly determine which children are enrolled).
In order to be able to generalize the findings to solid Montessori primary programs in the public sector beyond those that participated in the study, a diverse sample of schools would need to be recruited.7
In order to have the statistical power to test the impact of Montessori primary education across racial and income groups, large enough samples of each group of interest would need to be recruited and maintained for all three years of data collection.
The primary goals of the study and these methodological issues informed the criteria that were used to determine which schools we might approach to partner with us to conduct the study. These included:
Implementing a solid Montessori primary program (regardless of teacher training affiliation);
Being in the public sector and tuition free for families (either district, charter, or magnet in order to inform public early education efforts);
Beginning at age three (in order to study the full primary level, serving children PreK3 through kindergarten);
Admitting children via a lottery in which more families apply for placements than are available (in order to conduct an RCT); and
That across schools, a diverse sample of children could be recruited (in order to test impacts by race and family income).
As the BEFMIN researched schools to potentially partner with, it became clear that conducting an RCT might come at the expense of obtaining a diverse sample of Montessori schools. Stating this from a research methods perspective, we began to realize that maintaining the internal validity of the study (being able to conclusively determine cause and effect) was perhaps coming at the expense of external validity (being able to generalize the results to all well-implemented public Montessori primary programs in the U.S.). As noted above, to conduct an RCT, we needed to partner with schools that had more applicants than available placements and used lotteries to randomly determine enrollment, as only children who were randomly awarded placements in the program or placed on the waitlist would be eligible to participate in the study. Due to this criterion, a large number of Montessori primary programs in the public sector could not be considered for the study as they do not use lotteries, such as those that enroll on a first come first serve basis.
Further, we also discovered that many schools that include the use of lotteries in their enrollment policies actually admit few if any students randomly via a lottery and thus did not meet the study criteria. We found a variety of reasons contributed to this including other enrollment policies (such as preferences for neighborhood children or siblings) or that the program had fewer applicants than available placements (thus a lottery was not necessary). Another important criterion was that children be admitted via a lottery at age three in order to study the full Montessori primary cycle (PreK3-K). We found that many schools began at age four and/or admitted younger children but held the lottery at age five.
Thus, requiring that children be admitted at age three via a randomized lottery and that schools have a robust waiting list to serve as the control group resulted in a small number of schools even available to approach to partner with us, likely limiting the ability to generalize the findings to all schools in the public sector that implement solid Montessori primary programs.
Despite the fact that there were fewer schools that met the study criteria than we had originally expected, we were able to recruit a number of schools that did. BEF recognizes how significant the request to participate in this study was and is very grateful to the schools and school systems that generously agreed to partner with the network to investigate these critical questions. Even if these schools may not have been fully representative of the diverse types of public Montessori primary programs, we recognized and were willing to accept the potential limitation to the generalizability of the findings.
However, the initiative confronted another problem that became increasingly difficult to overcome, that of having adequate statistical power to investigate the primary aims of this study. Power analyses determined that the sample size needed to test the overall impact of Montessori was considerable (to account for the fact that children would be grouped within classrooms within schools). Given the primary goal of testing the impact of the Montessori approach across racial and economic groups, the sample size required for this study needed to be at least three times that needed to test for an overall impact (i.e., three racial groups, each requiring a large enough sample to test impact for that group). In addition, the populations needed to be recruited for the study tend to be those most difficult to retain in a study over time for a variety of reasons (such as higher mobility).
Thus, the power analyses indicated that a very large sample would be needed to conduct the study. This, along with the limited number of schools that could be recruited into the study given the issues concerning lotteries noted above raised serious concerns.
The first wave of data collection revealed more challenges. Fewer families were recruited than expected for a number of reasons. First, we continued to face challenges concerning lotteries, and many children (often the majority at a site) were not assigned placements via a true lottery. At some sites, this was due to other policies such as sibling and neighborhood preferences. At other sites, there was no need for an actual lottery to take place, and there were few or no children left on the waitlist to serve as the control group. Often, these were schools in which the majority of children attending were children of color (resulting in us not being able to include those schools in the study), while other schools in which the majority of children were White did have long waitlists.
Another challenge the researchers faced was that given the current political climate, Latine families were understandably reluctant to participate in a study that would require they provide personal information regardless of assurances that all data would be kept confidential. Combined, these challenges resulted in a decreased ability to recruit the very families needed to address the aims of the study.
Thus, the realization of the truly large numbers needed to have the statistical power to meet the aims of the study combined with the challenges faced recruiting diverse families from schools that met all the study criteria caused the BEF Board to arrive at the difficult conclusion that the study as originally conceived was not feasible.
Make the Problem the Solution
As noted above, the primary goal of the BEF Montessori Initiative is to inform efforts to close the racial and economic disparities in educational achievement and success. When it became clear that the original study was not feasible, the BEF Board deliberated on how to move forward, building upon the important lessons learned from attempting to conduct this study, and still make a significant contribution to the field.
It appears that the systems that create and maintain inequities in early education, and education writ large, may also make it difficult to rigorously study their effects. The difficulties the teams confronted in trying to conduct a study with a strong research design revealed challenges in the educational systems serving children of color and families with limited economic resources. That is, the work done attempting to investigate the effectiveness of a non-traditional educational approach may have uncovered some of the challenges many families face gaining access to these educational settings.
The Board concluded that the solution to how to inform efforts to close racial and economic disparities in education is to conduct an in-depth study focused on challenges to access to public Montessori early education programs.
Recognizing the strong network that was created to conduct the study, the BEF Board developed a way forward that will focus on the policies and practices that form these systems, address the original aims of investigating whether Montessori education is equally effective for children raised in diverse cultural and economic contexts, and help to generate further work in this important area. In this new direction, rather than working on one multi-year study, the teams will work as a true network conducting related, but separate, projects that contribute to the underlying goals of the Initiative.
Child Trends Policy Study
The team at Child Trends, in collaboration with Furman University, will conduct a study that will gather more detailed information about the systemic challenges that were revealed when attempting to do the original study.8 The research teams found that while some schools and districts appear to have policies in place that promote access to desired educational settings for all families, others may have policies that either overtly or covertly restrict it to historically advantaged groups.
In almost all areas, free early childhood education for three-year-old children is scarce. So, by definition, decisions about the physical location of this scarce resource affects which families can access it. Further, in some locations that intend to give parents a choice concerning their children’s education and use a lottery to assign children to schools when there are more applicants than available placements, in reality few if any children gain access via the lottery due to other policies, such as placement priority being given to families with siblings already enrolled or priorities based on neighborhood preferences. As documented in other work, often these neighborhood preference policies are put into place to protect access for families of color and those with low economic resources living in the neighborhood around the school.9However, as gentrification changes neighborhood demographics, the policies that were intended to protect access may result in restricting it. Thus, although the intent is to give parents the opportunity to make a choice concerning their children’s education, other policies and practices may result in fewer families of color and those with fewer resources having such an opportunity.
This team will conduct a public policy/landscape study to investigate the extent to which access to public Montessori pre-K programs is equitable. More specifically, the study will focus on the following research questions:
What are the recruitment practices and enrollment policies of public pre-K Montessori programs?
In public schools offering Montessori pre-K, what are the factors that contribute to programs having more or fewer applicants than available placements?
What are families’ perceptions of Montessori pre-K programming and other early education options?
Riley Institute at Furman University QED Study
The team at Furman University will conduct a study to address the original goal of investigating whether Montessori education has the potential to close the opportunity gaps.10 They will conduct a quasi-experimental study using administrative data to inform whether Montessori education in the public sector is equally effective across racial and income groups, focusing on achievement (standardized test scores), behavioral (disciplinary referrals and suspensions), and engagement (attendance rates) outcomes in pre-k/K and older grades.
This work will be similar to the study they conducted concerning Montessori public education in South Carolina but will also include other areas of the country. Although this will not be an RCT, it will have enough power to statistically control for some of the factors that are associated with selecting non-traditional school settings and thus make an important contribution to the field.
The What Works Clearinghouse (WWC), an initiative of the Institute of Education Sciences (IES) to provide educators and policymakers a source for information to make evidence-based decisions in education, has established a set of rigorous standards for research methodology used to investigate the effectiveness of curricula and interventions.11 While only RCTs are eligible for the highest rating of meeting standards without reservations, IES has concluded that rigorous quasi-experimental designs can also be used to investigate effectiveness, with reservations, when specific standards are met (such as establishing that the groups are similar before the intervention, which in this case is Montessori education).
The team at Furman University has extensive expertise with these methods and will strive to follow these standards as they conduct this study. When several research studies using a variety of designs reach similar conclusions, the convergence of evidence across them can be quite compelling. Thus, the findings from this work, when considered with those from other rigorously designed studies, will provide further understanding of the effectiveness of Montessori education for children of color and children raised in families with low economic resources.
University of Kansas Center for Montessori Research Activities
The team at the University of Kansas will engage in two activities that have the potential to contribute to future work in this area: continuing their measurement development work and holding their next annual Montessori Research Retreat with an enhanced focus on equity.12
This team will continue their work of developing a classroom observation instrument that will enable individuals without Montessori training to identify elements of well-implemented Montessori education, thus allowing researchers to account for variability in Montessori environments in their analysis.
BEF is also supporting their next annual Montessori Research Retreat that will have an enhanced focus on equity, thus potentially stimulating further research projects that will draw upon the strengths that children of color bring to the classroom and meet the needs of diverse families given the challenges they confront in our society in general and navigating the public educational systems in this country in particular.
As the work of the network continues is this new direction, the BEF Board will stay abreast of findings from a study being led by the American Institutes for Research (AIR), in partnership with the University of Virginia, investigating the overall effects of primary Montessori education in the public sector. Just as BEF launched its initiative, AIR was awarded a grant from the U.S. Department of Education, Institute of Educational Sciences (IES) to investigate the impact of Montessori education in the public sector on children’s academic and social-emotional outcomes, and this study has the potential to provide much needed information concerning scaling up the Montessori approach in the public sector. The AIR study and the original BEFMIN study were beginning at the same time, and the BEF Board is grateful for the many ways in which the AIR researchers collaborated with us.
Greater understanding of an issue is provided by exploring a variety of related questions using different methodologies, and the results that will be obtained from the AIR and BEFMIN studies will all contribute to a body of literature that can inform public early education efforts.
The opportunity and resulting achievement gaps are one of the pernicious social problems in the U.S., requiring diverse expertise and experiences to find solutions. The success of this work, and subsequently policy drivers, depends on authentic collaborations across disciplines, practice, cultures and ethnicities, school systems, and geographic locations.
The BEF Board is grateful for the strong network that came together to address the goals of this initiative, the collaboration of AIR research team, and most importantly the schools and school systems that were willing to partner with us to do this important work.
Confronting the challenges that the BEFMIN faced when attempting to conduct the original study was difficult, but they also revealed other important work that is needed. This new direction, although an adjustment, is consistent with the ultimate goal of the initiative to generate evidence that will inform the philanthropic community and policymakers about effective ways to eliminate racial and economic disparities, while also ensuring educational excellence.
1 Burchinal, M., McCartney, K., Steinberg, L., Crosnoe, R., Friedman, S. L., McLoyd, V., . . . Network, N. E. C. C. R. (2011) Examining the Black–White Achievement Gap Among Low-Income Children Using the NICHD Study of Early Child Care and Youth Development. Child Development, 82(5), 1404-1420. doi:10.1111/j.1467-8624.2011.01620.x; Duncan, G. J., Dowsett, C. J., Claessens, A., Magnuson, K., Huston, A. C., Klebanov, P., . . . Japel, C. (2007) School readiness and later achievement. Developmental Psychology, 43(6), 1428-1446. doi:10.1037/0012-16188.8.131.528; Duncan, G. J., & Magnuson, K. A. (2005) Can family socioeconomic resources account for racial and ethnic test score gaps? The Future of Children, 15(1), 35-54. doi:10.1353/foc.2005.0004; Gregory, A., Skiba, R. J., & Noguera, P. A. (2010) The achievement gap and the discipline gap: Two sides of the same coin? Educational Researcher, 39(1), 59-68.; Reardon, S. F. (2013) The widening income achievement gap. Education Leadership, 70(8), 10-16.; Reardon, S. F., Weathers, E.S., Fahle, E.M., Jang, H., & Kalogrides, D. (2019) Is Separate Still Unequal? New Evidence on School Segregation and Racial Academic Achievement Gaps (CEPA Working Paper No.19-06). Stanford, CA: Stanford Center for Education Policy Analysis.
2 Welner, K. G. & Carter, P. L. (2013) Achievement Gaps Arise from Opportunity Gaps. In P. L. Carter and K. G. Welner (Eds.), Closing the Opportunity Gap: What America Must Do to Provide Every Child and Even Chance. Oxford: Oxford University Press.
3 Gregory, A., Skiba, R. J., & Noguera, P. A. (2010) The achievement gap and the discipline gap: Two sides of the same coin? Educational Researcher, 39(1), 59-68.; Ladson-Billings, G. (2006) From the Achievement Gap to the Education Debt: Understanding Achievement in U.S. Schools. Educational Researcher, 35(7), 3-12.; Parsons, E. C., & Turner, K. (2014) The Importance of History in the Racial Inequality and Racial Inequity in Education: New Orleans as a Case Example. Negro Educational Review, 65(1-4), 99-113.; Talbert-Johnson, C. (2004) Structural Inequities and the Achievement Gap in Urban Schools. Education and urban society, 37(1), 22-36. doi:10.1177/0013124504268454.
4 Lillard A. S., Heise M. J., Richey E. M., Tong X., Hart A. & Bray P. M. (2017) Montessori Preschool Elevates and Equalizes Child Outcomes: A Longitudinal Study. Front. Psychol. 8:1783. doi: 10.3389/fpsyg.2017.01783.
5 Culclasure, B., Fleming, D.J., Riga, G., & Sprogis, A. (2018). An Evaluation of Montessori Education in South Carolina’s Public Schools. The Riley Institute at Furman University. Unpublished manuscript.
6 Consistent with experts in the field, we use Latine to refer to individuals whose cultural background originated in Latin America. In U.S. academic circles, Latinx is being used as a gender-inclusive term to refer to people from Latin American backgrounds, but Spanish-speakers find that Latinx is unpronounceable in Spanish. Therefore, we have opted to use the gender-inclusive term Latine, commonly used throughout Spanish-speaking Latin American countries. Melzi, G., Prishker, N., Kawas, V., Huancacuri-Harlow, J. (Forthcoming). Multilingual Parenting in the United States: Language, Culture and Emotion. In A. Stavans and U. Jessner (Eds.), The Cambridge Handbook of Childhood Multilingualism. Cambridge: Cambridge University Press.
7 Primary Montessori programs in the public sector that are similar with respect to implementing a solid practice can differ in a number of respects including such factors as geographic location, school type (e.g., charter, magnet, district schools; stand-alone schools, school-within-a-school), teacher training, and Montessori accreditation.
8 The team at Child Trends includes Diane Early, Joy Thompson, Danielle Hegseth, Porsche Boddicker-Young, and Rowan Hilty; and the researcher at Furman University collaborating with Child Trends on this study is Delia Allen.
9 Debs, M. (2019). Diverse Families, Desirable Schools: Public Montessori in the Era of School Choice. Cambridge, MA: Harvard Education Publishing Group.
10 The team at Furman University includes Brooke Culclasure, David Fleming, Delia Allen, Hall West, and Alexis Sprogis.
11 U.S. Department of Education, Institute of Education Sciences, National Center for Education Evaluation and Regional Assistance, What Works Clearinghouse. Standards Handbook. Version 4.0 (https://ies.ed.gov/ncee/wwc/Docs/referenceresources/wwc_standards_handbook_v4.pdf)
12 The team at the University of Kansas includes Angela Murray, Jie Chen, and Carolyn Daoust.
Additional Resources Concerning Montessori
The Montessori Comeback in Forbes by Emily Langhorne
The Montessori Preschool Landscape in the United States: History, Programmatic Inputs, Availability, and Effects, an ETS Research Report by Debra J. Ackerman