A Deeper Look at Marzano

The expensive and rigorous Marzano teacher training programs are based on a model that consists of 60 elements grouped into four domains:

            Domain 1: Classroom Strategies and Behaviors

            Domain 2: Planning and Preparing

            Domain 3: Reflecting on Teaching

            Domain 4: Collegiality and Professionalism

The first domain, Classroom Strategies and Behaviors, is the one that most directly affects students; it comes down to what goes on in the classroom during class time. The 41 elements it consists of are categorized into three segments: routine events, content, and actions on the spot. From there, each segment has a series of design questions that use specific elements as guidelines for implementation. For example, a design question from the first segment “routine events” asks the teachers what they would do to “establish and communicate learning goals, track student progress, and celebrate success.” The model would then list three elements to answer each question. Elements would include “providing clear learning goals and scales,” “tracking student progress,” and “celebrating success.” It would then be up to the teacher to find ways to bring these out through her class dynamic.

The second domain, Planning and Preparing, is simply based on the idea that thorough planning and preparation will give teachers enough time to implement the recommended strategies in the classroom. It includes preparation for lessons and units, preparation for the use of materials and technology, and preparation for any special needs of students. The third domain, Reflecting on Teaching, comprises only of teacher self-evaluation, growth plan development, and implementation. Finally, the fourth domain, Collegiality and Professionalism, focuses on fostering a healthy and cooperative teaching environment.

The purpose of the Marzano Model is to help students learn by providing a basis for teacher evaluation and professional development in schools. Instead of relying solely on student test scores to evaluate teachers, the Marzano Model offers a different approach to observe teachers in a fairer, more comprehensive way. The idea is that incremental improvements  in teaching would “cause” increases in student achievement. We have all had those teachers that confuse us way more than they actually help us; we cannot argue against the fact that there needs to be some sort of standard of competence for teachers in our country, as there is quite a disparity in the quality of education in the states. The Marzano Model is a product of over 5,000 studies spanning the last five decades. It has conducted over 300 experimental/control studies involving over 14,000 students and 300 teachers across 38 schools in 14 districts. Studies have shown an average increase in student achievement by 16 percentile points when teachers have implemented the elements of the model into their teaching. (The average effect size of .42 associated with the 16^th percentile point has seen increases to 2.00 in some studies.) No doubt the statistics are appealing to schools seeking to improve their ratings, and Marzano has all the credentials in the world to reinforce them.  But can we really trust these statistics?
 
Where does Marzano go wrong?

Let me start by saying that there have been no formal peer-reviews of Marzano’s findings.  Marzano largely bases his research off of meta-analysis: the combining of different statistical results from different studies in hopes of identifying a pattern.  Usually, meta-analysis uses studies throughout a number of years and, based on the validity of those studies, they are selected for inclusion.  The issue with Marzano is that his meta-analysis does not hold itself to convincingly strong criteria.  Marzano’s research team conducted a variety of small separate studies and then synthesized the findings to come to a formal conclusion.  It is unusual for meta-analysis to be done this way, with the same research team conducting each unit of study in a similar time frame.  In his research, Marzano used relatively small samples testing separate elements of his model.  Because the sample sizes were small, the overall effects of his program were magnified.  The primary way in which he conducted his studies was by asking teachers – each with two similar classes – to give their students a pretest and a posttest and gauge their improvement in a percentage.  One class was given the experimental variable while the other was not.  However, the internal validity of the experiment is questionable.  The teachers Marzano used for his research varied by subject, grade level, and state.   In addition,  there were psychological variables amongst the participants of the study that could have interfered.  Examples of psychological variables can be drawn from Marzano’s partnership with Promethean, which had the purpose of examining the effects of interactive whiteboard systems on learning (need I point out one of the reasons behind this study was to sell whiteboards?).  There could have been a rivalry between the students belonging to the class that used the technology and those belonging to the class that didn’t, and some students may have tried to purposefully thwart the results out of jealousy.  The teachers also knew what results Marzano was looking for, and paid better attention to their teaching and students.  As a result, there did not truly exist a dependent variable for comparison.  Finally, Marzano used percentiles incorrectly.  Percentages are not the same as percentiles.  Marzano used bell curves to obtain the value of the percentile gains he claims.  However, this does not work because the scores of the dependent variables (the performance of the student)  are not normally distributed (this is a statistical term that I encourage you to read more in depth on).

So should we entirely discredit his findings? I do not believe we should.  Just as there is a degree of uncertainty toward the validity of Marzano’s findings, there is also a degree of uncertainty toward the invalidity of his findings.  However, Marzano will have to do more research using more reliable methods before he wins us over entirely.  Only time (and more research) will tell.