Mapping the Architecture of Misconceptions: Network Analysis and Confidence Calibration in Three-Tier Diagnostic Assessment with Implications for Targeted Instruction

Abstract

Traditional science assessments inadequately distinguish between lack of knowledge and confident misconceptions, limiting their diagnostic value for instructional design. This study developed and validated a three-tier multiple choice diagnostic instrument for Indonesian students studying ecology and biodiversity concepts, integrating content questions, reasoning probes, and confidence ratings using the Certainty of Response Index. We administered a 15-item instrument to 95 seventh-grade students at a public junior high school in Central Java, Indonesia. Validation employed multi-method approaches including classical test theory, Item Response Theory modeling, confidence calibration analysis, and network analysis of misconception co-occurrence patterns. Student responses were classified as Sound Understanding with Confidence (PKDY), Sound Understanding with Uncertainty (PKTTY), Misconceptions (M), or Lack of Knowledge (TPK). The instrument demonstrated acceptable reliability (Cronbach's α=0.647-0.682) and high content validity (0.833). Classification revealed 37.4% PKDY, 33.9% misconceptions, 19.0% lack of knowledge, and 9.7% uncertain understanding. Students exhibited systematic overconfidence (OI=0.024) with Dunning-Kruger effects. Network analysis identified four misconception communities with coherent clustering patterns rather than random errors. Two-parameter logistic IRT models provided superior fit compared to Rasch models. The methodology advances diagnostic assessment by revealing which misconceptions function as conceptual bridges in student thinking, providing actionable intelligence for targeted instruction. For educational practice, the identification of 33.9% confident misconceptions—which traditional binary systems would simply mark as "incorrect"—demonstrates substantial gains in diagnostic precision essential for effective remediation. The framework offers replicable protocols for developing culturally responsive diagnostic tools with implications for teacher professional development and curriculum sequencing in diverse educational contexts.