Validation of the evaluation instrument: competencies of the institutional project
The text shows a way to approach the evaluation with a competency approach, using an instrument designed with balanced complete blocks, which are rotated in a spiral to assemble different test booklets. The construct that is evaluated is the knowhow, with the knowledge of a field, in different contexts of the world of life, which is derived from the understanding of the Institutional Project, PI, of the Cooperative University of Colombia.
Know-how, understood as the ability to perform tasks that are organized into four levels of the SOLO (Structure of Observed Learning Outcomes) taxonomy proposed by John Biggs: unistructural, multistructural, relational, and extended abstract. Knowing about a field, in terms of knowledge and mastery of the concepts and elements, with which the PI statements are structured, about the mission and vision, the principles and values, the teacher, self-assessment and accreditation, the social projection and extension, curricular guidelines, management, welfare and internationalization, among others. And contexts of the world of life, in the form of fragments that illustrate local, social or global situations and that can be placed in dialogue with the object of evaluation, such as: Teachers, by Alfonso Borrero, Declaration for the design of the European Education System , by La Sorbonne, About communicative competence, by Dell Hymes, Education for a new humanism, by Guillermo Hoyos, Contemporary forms of government and university administration, by Miguel Ángel Escotet, or Universities as producers of knowledge in Colombia, by Milcíades Vizcaino and Ruth Muñoz, among others.
The main contribution of the text is the validation of the instrument used, which comprises two central phases. First, content validation, which takes place within a college of teachers and administrators, before application, based on the theoretical discussion and the argued consensus of each component, to ensure the pertinence and relevance of the situations chosen in the contexts, the statements taken from the IP as a reference and the statements and options of the items or questions formulated, according to the levels of competence established in the design. And second, statistical validation, of an empirical nature, which occurs after the application, which is carried out by estimating indicators that arise when solving the Rasch model, of the item response theory with one parameter, that of difficulty, taking as input the universe of response chains achieved, to analyze the observed performance and confirm the validity, both of the items individually and independently and of the instrument as a whole.
The values estimated in the statistical validation phase, and which are presented in the text together with some acceptance criteria recommended in the respective state of the art, support the quality of the instrument developed and applied, as well as the reliability of the results found, to effects of making certain decisions and supported by accurate information. On the other hand, the text provides an adequate methodology to demonstrate the statistical validity of evaluation instruments.
