In this work, we assess whether and to what extent the latest Italian national policy initiatives... more In this work, we assess whether and to what extent the latest Italian national policy initiatives intended, among others, to contrast favoritism and foster recruitment effectiveness have resulted in the desired effects. To answer the question, we propose two related analyses. One compares the research performance ratings of recruits in two subsequent five-year periods, before and after the introduction of the above policy measures. The second analysis compares the effectiveness of recruitment by all Italian universities, in the above two subsequent five-year periods. The results from the comparisons show a decline of both unproductive and high-performing recruits, and no overall improvement in the effectiveness of recruitment.
In this work, we assess whether and to what extent the latest Italian national policy initiatives... more In this work, we assess whether and to what extent the latest Italian national policy initiatives intended, among others, to contrast favoritism and foster recruitment effectiveness have resulted in the desired effects. To answer the question, we propose two related analyses. One compares the research performance ratings of recruits in two subsequent five-year periods, before and after the introduction of the above policy measures. The second analysis compares the effectiveness of recruitment by all Italian universities, in the above two subsequent five-year periods. The results from the comparisons show a decline of both unproductive and high-performing recruits, and no overall improvement in the effectiveness of recruitment.
In this work, we try to answer the question of which method, peer review vs bibliometrics, better... more In this work, we try to answer the question of which method, peer review vs bibliometrics, better predicts the future overall scholarly impact of scientific publications. We measure the agreement between peer review evaluations of Web of Science indexed publications submitted to the first Italian research assessment exercise and long-term citations of the same publications. We do the same for an early citation-based indicator. We find that the latter shows stronger predictive power, i.e., it more reliably predicts late citations in all the disciplinary areas examined, and for any citation time window starting one year after publication.
This is the first ever attempt of application in a country other than Italy of a research efficie... more This is the first ever attempt of application in a country other than Italy of a research efficiency indicator (FSS), to assess and compare the performance of professors and universities, within and between countries. A special attention has been devoted to the presentation of the methodology developed to set up a common field classification scheme of professors, and to overcome the limited availability of comparable input data. Results of the comparison between countries, carried out in the 2011-2015 period, show similar average performances of professors, but noticeable differences in the distributions, whereby Norwegian professors are more concentrated in the tails. Norway shows notable higher performance in Mathematics and Earth and Space Sciences, while Italy in Biomedical Research and Engineering.
In this work, we assess whether and to what extent the latest Italian national policy initiatives... more In this work, we assess whether and to what extent the latest Italian national policy initiatives intended, among others, to contrast favoritism and foster recruitment effectiveness have resulted in the desired effects. To answer the question, we propose two related analyses. One compares the research performance ratings of recruits in two subsequent five-year periods, before and after the introduction of the above policy measures. The second analysis compares the effectiveness of recruitment by all Italian universities, in the above two subsequent five-year periods. The results from the comparisons show a decline of both unproductive and high-performing recruits, and no overall improvement in the effectiveness of recruitment.
In this work, we assess whether and to what extent the latest Italian national policy initiatives... more In this work, we assess whether and to what extent the latest Italian national policy initiatives intended, among others, to contrast favoritism and foster recruitment effectiveness have resulted in the desired effects. To answer the question, we propose two related analyses. One compares the research performance ratings of recruits in two subsequent five-year periods, before and after the introduction of the above policy measures. The second analysis compares the effectiveness of recruitment by all Italian universities, in the above two subsequent five-year periods. The results from the comparisons show a decline of both unproductive and high-performing recruits, and no overall improvement in the effectiveness of recruitment.
In this work, we try to answer the question of which method, peer review vs bibliometrics, better... more In this work, we try to answer the question of which method, peer review vs bibliometrics, better predicts the future overall scholarly impact of scientific publications. We measure the agreement between peer review evaluations of Web of Science indexed publications submitted to the first Italian research assessment exercise and long-term citations of the same publications. We do the same for an early citation-based indicator. We find that the latter shows stronger predictive power, i.e., it more reliably predicts late citations in all the disciplinary areas examined, and for any citation time window starting one year after publication.
This is the first ever attempt of application in a country other than Italy of a research efficie... more This is the first ever attempt of application in a country other than Italy of a research efficiency indicator (FSS), to assess and compare the performance of professors and universities, within and between countries. A special attention has been devoted to the presentation of the methodology developed to set up a common field classification scheme of professors, and to overcome the limited availability of comparable input data. Results of the comparison between countries, carried out in the 2011-2015 period, show similar average performances of professors, but noticeable differences in the distributions, whereby Norwegian professors are more concentrated in the tails. Norway shows notable higher performance in Mathematics and Earth and Space Sciences, while Italy in Biomedical Research and Engineering.
Uploads