How effective are educational systems? A value-added approach to study trends in PIRLS

Journal article

From an educational effectiveness perspective, research based on international large scale assessments has been limited as it neglects to take contextual conditions of educational systems into account. Further, methodological challenges of cross-sectional studies have yet prevented investigations from a longitudinal effectiveness perspective. The paper investigates how effectively educational systems grow, i.e. change, in their performance by applying a methodological approach known from school effectiveness research that captures changes at the country level within repeated cross-sectional data designs. Data from the Progress in International Reading Literacy Study (PIRLS) 2001 to 2006 trend systems is analyzed with hierarchical linear modeling. Effectiveness measures of achievement status in 2006 and of change from 2001 to 2006 are investigated and compared. Results suggest that there are systems which exceed their expected outcomes (status and change) as well as systems which stay below what could have been expected, changing the picture of “high” and “low” performing systems, when contextual conditions and prior performances are taken into account. The study contributes to methodological developments of educational effectiveness research in cross-national assessments. Its results provide complementary information for policymakers to further look at policies, practices, and structures that have favored effectiveness.

Authors: Lenkeit, J

• Alternative title: Wie effektiv sind Bildungssysteme? Zur Untersuchung von Entwicklungen in PIRLS mit value-added-Modellen
• Publication status: Published
• Peer review status: Peer reviewed
• Publisher: Waxmann
• Journal: Journal for Educational Research Online / Journal für Bildungsforschung Online
• Volume: 4
• Issue: 2
• Pages: 144-173
• Publication date: 2013-02-01
• Acceptance date: 2013-02-25
• EISSN: 1866-6671
• Language: English
• Keywords: educational effectiveness; cross-national comparisons; repeated cross-sectional design