Evaluation of Validity and Validation by Means of the Argument-based Approach

Validity is the most important quality aspect of tests and assessments, but it is not clear how validity can be evaluated. This article presents a procedure for the evaluation of validity and validation which is an extension of the argument-based approach to validation. The evaluation consists of three criteria to evaluate the interpretive argument, the validity evidence provided, and the validity argument. This procedure is illustrated with an existing assessment: the driver performance assessment. The article concludes with recommendations for the application of the procedure. Keywords Competence assessment, validity, validation, argument-based approach, evaluation. Expressive Reading Aloud on IEA-PIRLS 2006 Gabriella Agrusti This in-depth research study was aimed at investigating the effects of readaloud modification on students’ performance on PIRLS 2006 reading comprehension tests, in two different forms: expressive reading and neutral reading. In Italy international comparative surveys often represent the main reference measure for student achievements in basic skills, but few experimental designs descending from secondary analyses attempt to investigate possible relationships among variables in order to translate results in suggestions for teaching practices. The present study was intended as a first step in this direction, analyzing if specific aspects of reading aloud can influence students’ achievements in comprehension. Differences in means between groups were found statistically significant between expressive reading aloud administration and silent standard administration, though the strong concurrent validity of PIRLS 2006 test items allowed only a small variance in results due to experimental modification. Neutral reading aloud alone did not have significantly different effects on test results, confirming the general assumption that at this age word recognition skill is at an advanced level. More evident results were shown in items focused on the processes of interpreting and integrating ideas and information, highlighting how inferential more than retrieval processes are influenced by reading aloud, that efficiently convey the gist of the story and some of the implications in meaning of complex semantic topics, such as the identification and description of main characters’ feelings.

Keywords: PIRLS 2006, reading comprehension, expressive reading aloud, neutral reading aloud, assessment.

• Constructing validity arguments for test combinations Saskia Wools, Theo J.H.M. Eggen, Anton A. Béguin, in Studies in Educational Evaluation /2016 pp.10
  DOI: 10.1016/j.stueduc.2015.11.001
• Developing questionnaires to measure students’ expectations and perceptions of assessment quality Karin J. Gerritsen-van Leeuwenkamp, Desirée Joosten-ten Brinke, Liesbeth Kester, Sammy King Fai Hui, in Cogent Education /2018 pp.1464425
  DOI: 10.1080/2331186X.2018.1464425
• Uses of Technology in Lower Secondary Mathematics Education Paul Drijvers, Lynda Ball, Bärbel Barzel, M. Kathleen Heid, Yiming Cao, Michela Maschietto, pp.1 (ISBN:978-3-319-33665-7)
• Theoretical and Practical Advances in Computer-based Educational Measurement Saskia Wools, Mark Molenaar, Dorien Hopster-den Otter, pp.3 (ISBN:978-3-030-18479-7)
• Factors influencing assessment quality in higher vocational education Liesbeth Baartman, Judith Gulikers, Asha Dijkstra, in Assessment & Evaluation in Higher Education /2013 pp.978
  DOI: 10.1080/02602938.2013.771133
• VAST: a practical validation framework for e-assessment solutions Max van Haastrecht, Matthieu Brinkhuis, Saskia Wools, Marco Spruit, in Information Systems and e-Business Management /2023 pp.603
  DOI: 10.1007/s10257-023-00641-3
• Digital assessment of mathematics: Opportunities, issues and criteria Paul Drijvers, in Mesure et évaluation en éducation /2019 pp.41
  DOI: 10.7202/1055896ar
• The design, development, and validation of a multimedia-based performance assessment for credentialing confined space guards Sebastiaan de Klerk, Bernard P. Veldkamp, Theo J. H. M. Eggen, in Behaviormetrika /2018 pp.565
  DOI: 10.1007/s41237-018-0064-x
• Serious games in high-stakes assessment contexts: a systematic literature review into the game design principles for valid game-based performance assessment Aranka Bijl, Bernard P. Veldkamp, Saskia Wools, Sebastiaan de Klerk, in Educational technology research and development /2024 pp.2041
  DOI: 10.1007/s11423-024-10362-0
• Assessment quality in tertiary education: An integrative literature review Karin J. Gerritsen-van Leeuwenkamp, Desirée Joosten-ten Brinke, Liesbeth Kester, in Studies in Educational Evaluation /2017 pp.94
  DOI: 10.1016/j.stueduc.2017.08.001
• A General Framework for the Validation of Embedded Formative Assessment Dorien Hopster‐den Otter, Saskia Wools, Theo J. H. M. Eggen, Bernard P. Veldkamp, in Journal of Educational Measurement /2019 pp.715
  DOI: 10.1111/jedm.12234

Saskia Wools, Evaluation of Validity and Validation by Means of the Argument-based Approach in "CADMO" 1/2010, pp 63-82, DOI: 10.3280/CAD2010-001007