CEMSE’s work on fidelity of implementation grew out of a National Science Foundation- supported project called "Applying Research on Science Materials Implementation: Bringing Measurement of Fidelity of Implementation (FOI)to Scale." That project, which we refer to as the "FOI Project," set out to create instruments to measure the fidelity of implementation of reform-based science and mathematics instructional materials at the K-8 level. Over three years, the project team worked in collaboration with the Chicago Public Schools to produce instruments for measuring FOI of five specific curricula (Full Option Science System (FOSS), Science and Technology for Children (STC), Science Companion, Science Education for Public Understanding Program, and Everyday Mathematics).
NSF Project Documents
Throughout the FOI project, CEMSE staff produced two project briefs and four technical reports that describe the project approach and strategies for addressing substantive and technical issues related to measuring the use of instructional materials. Additionally, the FOI Project staff has presented a number of papers on the FOI project. The most comprehensive paper, titled “A Framework for Measuring Fidelity of Implementation: A Foundation for Shared Language and Accumulation of Knowledge," was published in the American Journal of Evaluation in 2010.
Although the project focused on a set of instruments that could be used across these particular reform-based science and mathematics instructional materials, we found that the framework and its approach of using critical components as measurable constructs could apply to other mathematics and science instructional materials, and with adaptation, to other kinds of educational interventions.
Instruments for Measuring Fidelity of Implementation
The FOI Project has created a suite of instruments (teacher instructional questionnaire, teacher attitude questionnaire, teacher instructional log, teacher interview protocol, school leader questionnaire, school leader interview protocol, school wide observation protocol, classroom observation protocol) for measuring what we used to refer to as fidelity of implementation (FOI) of instructional materials and now refer to as enactment or use of instructional materials. These materials measure a set of constructs that are derived from the "critical components" or essential elements of instructional materials. These critical components were identified through a process of review by the researchers, the developers, and users and is described in one of the technical reports mentioned in the previous section.
Information on the instruments, the constructs measured, psychometrics, and guidance for use are all available on line in CEMSE’s FOI User's Guide. We have housed the User’s Guide at a site called Researchers Without Borders (RWB). RWB is an initiative of CEMSE and is described below.
Other CEMSE Resources and Information on Fidelity of Implementation
Researchers Without Borders is a home for open research in education. It is a collaborative working environment in which individuals across academic institutions, disciplines, sectors and levels of organizational structure can direct their efforts to solve shared problems, do collaborative research and development and build productive working relationships and collaborations. Additional information on Fidelity of Implementation and CEMSE’s further efforts on measuring enactment of innovations can be found at Researchers Without Borders.
In addition to the FOI User’s Guide, RWB also has a set of FAQs about fidelity of implementation and a searchable database with many sources about fidelity of implementation, some of which are summarized and reviewed by CEMSE staff and RWB users. RWB also has an on-line community with a group of participants focused on fidelity of implementation.
For more information about the FOI project, contact Jeanne Century.