TITLE: A Modeling Framework for Analyzing the Education System as a Complex System

STUDENT: Pratik Mital

ABSTRACT:

In this thesis, the Education System Intervention Modeling Framework (ESIM Framework), is introduced for analyzing interventions in the K-12 education system. This framework is the first of its kind to model interventions in the K-12 school system in the United States. Techniques from systems engineering and operations research, such as agent-based modeling and social network analysis, are used to model the bottom-up mechanisms of intervention implementation in schools. By applying the ESIM framework, an intervention can be better analyzed in terms of the barriers and enablers to intervention implementation and sustainability.  The risk of failure of future interventions is thereby reduced through improved allocation of resources towards the system agents and attributes which play key roles in the sustainability of the intervention.

In the first part of this thesis, a case study is modeled which helped in the development of the framework. This case study was of an extracurricular school intervention, an Engineers Without Borders chapter, implemented in a magnet school setting through a partnership with Georgia Institute of Technology, Atlanta (Georgia Tech) as part of a National Science Foundation (NSF) GK-12 grant. This case study is ideal for the development of the framework and as its first application because it had two different outcomes over two different years, which helped in developing insights about the success of this intervention. Also, the scale of this intervention was small enough to test the development and application of the framework. With the help of this case study, a more generalized framework is developed which is applicable across a broad range of education system interventions.
In the second part of this thesis, the ESIM framework is developed. The framework developed is divided into four phases: model definition, model design, model analysis, and model validation. In the model definition phase, the overview of the problem to be modeled is documented. Then, detailed descriptions about the agents, attributes, and the environment being modeled are provided. Other modeling decisions, such as scale and time horizons, are also made in this phase. Finally, the criteria for a sustainable intervention is defined along with a method to analyze the risk of implementing this intervention in the particular school system. In the model design phase, the conceptual model is built using agent-based modeling, social network analysis, and discrete-time Markov chains. Then, the conceptual model is validated with the help of subject matter experts (SMEs) using Pace’s 4C’s framework for conceptual model validation. After that, the computer simulation model is implemented and verified. In the model analysis phase, simulation results are generated and analyzed. While simulating outcomes that are consistent with reality is helpful, the real contributions of this framework are two-fold: the sensitivity analysis of the model, and the determination of factors that are likely to affect the intervention outcomes.  The latter is accomplished using the Method of Morris, a factorial sampling technique. Finally, in the model validation phase, verification and validation techniques are applied. This step is critical in developing confidence in the model amongst its users.

In the third part of this thesis, the ESIM framework is applied to a case study of a curriculum intervention, Science Learning: Integrating Design, Engineering and Robotics, involving the design and implementation of an 8th-grade, inquiry-based physical science curriculum across three demographically varying schools. This intervention was also implemented in collaboration with Georgia Tech as part of an NSF DRK-12 grant. This was a five year intervention from Sep, 2009 to Oct, 2014. This case study provides a good comparison of the implementation of the intervention across different school settings because of the varied outcomes at the three schools.