Establishing Assessment Scales Using a Novel Disciplinary Rationale for Scientific Reasoning

Abstract

The article argues that science assessment should change from an item-driven to a construct-driven practice and pay more attention to disciplinary scientific reasoning. It investigates assessment scales developed from a novel theoretical rationale, describing scientific reasoning as three fundamental practices (hypothesizing, experimenting, and evidence evaluation) and building on three types of knowledge (science content knowledge, procedural knowledge, and epistemic knowledge). The scale development follows a construct-driven approach by, first, detailing the knowledge involved and explaining progression; and second, operationalizing the theoretical construct into items and score criteria. The scales are trialled in a small-scale study. The outcome is a coherent and supportive validity argument for two sub-scales, but with a suggestion that merging these into one scale has higher validity. The main implication is rewriting rationales for many science assessments, including TIMSS, which emphasises domain-general reasoning, and NAEP and PISA, which pay attention to domain-specific reasoning but are unclear about the knowledge involved.