Ricardo Conejo

Peer review is the "gold standard" for evaluating journal and conference papers, research proposals, on-going projects and university departments. However, it is widely believed that current systems are expensive, conservative and prone to various forms of bias. One form of bias identified in the literature is "social bias" linked to the personal attributes of authors and reviewers. To quantify the importance of this form of bias in modern peer review, we analyze three datasets providing information on the attributes of authors and reviewers and review outcomes: one from Frontiers - an open access publishing house with a novel interactive review process, and two from Spanish and international computer science conferences, which use traditional peer review. We use a random intercept model in which review outcome is the dependent variable, author and reviewer attributes are the independent variables and bias is defined by the interaction between author and reviewer attributes. We find no evidence of bias in terms of gender, or the language or prestige of author and reviewer institutions in any of the three datasets, but some weak evidence of regional bias in all three. Reviewer gender and the language and prestige of reviewer institutions appear to have little effect on review outcomes, but author gender, and the characteristics of author institutions have moderate to large effects. The methodology used cannot determine whether these are due to objective differences in scientific merit or entrenched biases shared by all reviewers.

ABSTRACT In this paper we present the implementation of a general mechanism to deliver tests based on mobile devices and matrix codes. The system is an extension of Siette, and has not been specifically developed for any subject matter. To evaluate the performance of the system and show some of its capabilities, we have developed a test for a second-year college course on Botany at the School of Forestry Engineering. Students were equipped with iPads and took an outdoor test on plant species identification. All students were able to take and complete the test in a reasonable time. Opinions expressed anonymously by the students in a survey about the usability of the system and the usefulness of the test were very favorable. We think that the application presented in this paper can broaden the applicability of automatic assessment techniques

ABSTRACT One of the most popular student modeling approaches is Constraint-Based Modeling (CBM). It is an efficient approach that can be easily applied inside an Intelligent Tutoring System (ITS). Even with these characteristics, building new ITSs requires carefully designing the domain model to be taught because different sources of errors could affect the efficiency of the system. In this paper a novel mechanism for studying the quality of the elements in the domain model of CBM systems is presented. This mechanism combines CBM with the Item Response Theory (IRT), a data-driven technique for automatic assessment. The goal is to improve the quality of the elements that are used in problem solving environments for assessment or instruction. In this paper we propose a set of statistical techniques, i.e., the analysis of the point-biserial correlation, the Cronbach’s alpha and the information function, to explore the quality of constraints. Two different tools have been used to test this approach: a problem solving environment designed to assess students in project investment analysis; and an independent component that performs assessments using CBM and IRT. Results suggest that the three methods produce consistent diagnosis and may be complementary in some cases. In the experiments we have carried out they were able to detect faulty, bad and good quality constraints.

ABSTRACT Most Adaptive and Intelligent Web-based Educational Systems (AIWBES) use tasks in order to collect evidence for inferring knowledge states and adapt the learning process appropriately. To this end, it is important to determine the difficulty of tasks posed to the student. In most situations, difficulty values are directly provided by one or more persons. In this paper we explore the relationship between task difficulty estimations made by two different types of individuals, teachers and students, and compare these values with those estimated from experimental data. We have performed three different experiments with three different real student samples. All these experiments have been done using the SIETTE web-based assessment system. We conclude that heuristic estimation is not always the best solution and claim that automatic estimation should improve the performance of AIWBES.

Seleccionar todos Título: Conferencia internacional sobre sistemas hipermedia y web adaptativos (AH2002) Autores: Conejo, Ricardo Revista: Inteligencia Artificial: revista iberoamericana de inteligencia artificial, 2002 OTO; (17) Página(s): 101-102 ISSN: 11373601. ...

RESUMEN En el presente trabajo introducimos una forma finitista de entender la calculabilidad fuzzy. En base a ella definimos la clase de W-funciones construibles. Demostramos que esta clase está estrictamente incluida en la clase de W-funciones con ...

Resumen. En este artículo proponemos el uso de redes bayesianas como marco teórico para la evaluación computerizada de los alumnos a partir de preguntas tipo test. Para ello, desarrollamos el modelo estructural que permitiría modelar el problema de los test adaptativos con una ...

Inteligencia Artificial. Revista Iberoamericana de Inteligencia Artificial. Asociación Española para la Inteligencia Artificial. revista@aepia.org. ISSN (Versión impresa): 1137-3601. ... No.12 (2001), pp. 50-58. ISSN: 1137-3601. © AEPIA (http://www.aepia.dsic.upv.es/). ...

Abstract: MOREA [1] was initially defined as a knowledge-based visual environment for the representation, simulation and calibration of hydrological basins. Thus, tasks that were manually accomplished by hydrologists, have been automated. The hydrological components are modeled by cognitive pieces with a behavior. This environment has been extended to validate hydrological information and to simulate different situations or hypotheses that may happen in the components of an hydrological basin.

Abstract. This paper presents a proposal of a framework for the integration of web-based educational systems. It is part of an on-going research work (MEDEA project), whose final goal is to develop a general framework to build open Intelligent Tutoring Systems (ITS). We understand “open system” as a set of autonomous educational modules which communicate between themselves, following high-level pre-established protocols.

Abstract. This paper introduces a set of resources that provide web learning environments with student modeling services. SAMUEL is a user modeling server for registering, updating and maintaining student knowledge data from different sources that use their own ontologies. In order to make inferences about student knowledge, it becomes necessary to establish equivalences between concepts of different domains.

The Item Response Theory (IRT) is a statistical mechanism successfully used since the beginning of the 20th century to infer student knowledge through tests. Nevertheless, existing well-founded techniques to assess procedural tasks are generally complex and applied to well-defined tasks. In this paper, we describe how, using a set of techniques we have developed based on IRT, it is possible to infer declarative student knowledge through procedural tasks. We describe how these techniques have been used with undergraduate students, in the object oriented programming domain, through ill-defined procedural exercises.

... Cesar Ferri Inmaculada Fortes Francisco-Jose Galan-Morillo Ana Garcfa-Sipols RM Gasca Adriana Giret-Boggino Daniela Godoy Manuel ... Huget Marc-Philippe J. Marques de Sa Paloma Martinez Jose Maria Martmez-Montiel Patricio Martmez-Barco Ignacio Mayorga Eva Millan ...

When a quantitative student model is constructed, one of the first tasks to perform is to identify the domain concepts assessed. In general, this task is easily done by the domain experts. In addition, the model may include some misconceptions which are also identified by these experts. Identifying these misconceptions is a difficult task, however, and one which requires considerable previous experience with the students. In fact, sometimes it is difficult to relate these misconceptions to the elements in the knowledge diagnostic system which feeds the student model. In this paper we present a data-driven technique which aims to help elicit the domain misconceptions. It also aims to relate these misconceptions with the assessment activities (e.g. exercises, problems or test questions), which assess the subject in question.

The introduction of systems for automatic data acquisition to monitor and control hydrological basins is a qualitative change in the field of hydrology. The large amount of information available increases the number of processes that can be analyzed with a quantitative approach. In the past, hydrological data validation was done manually by applying the knowledge of experts in the field. This article proposes to solve this problem using AI techniques. As a result, a generic model is defined for the cognitive task of data ...

This paper presents an application of Knowledge Engineering techniques to the problem of identifying a steel or cast iron from a microphotograph. The essential aim of the implemented system is to help metallography students in the task of learning the concepts relevant for identifying and classifying steels and cast irons. The system has been developed and implemented by means of Knowledge Engineering tools, and all the goals set up at the beginning of the project have been reached.

En este artıculo se presenta una propuesta para la introducción de mecanismos adaptativos para la elección de ayudas, en un entorno de evaluación basado en tests adaptativos. El artıculo comienza con una discusión sobre aspectos relativos a mecanismos de selección ...

In this paper we define some indicators that are useful to analyze the behavior of an individual within a learning community and hypothesize that there must exist some underlying factors relating to the user that explain the result obtained. We analyze the learner behaviors in the Comtella-D learning community and conclude that the actual data do not contradict our hypothesis.