independent evaluation
Recently Published Documents

The SARS-CoV-2 pandemic created a crucial need for accurate serology assays to evaluate seroprevalence, contact tracing, and antiviral immune responses. The initial flood of serology assays entering the market with inadequate performance emphasized the need for independent evaluation of commercial SARS-CoV-2 antibody assays using performance evaluation panels to determine suitability for use under emergency use authorization (EUA).

BACKGROUND Substance use and use disorders in the US have had significant and devastating impacts on individuals and communities, and the escalating substance use crisis calls for urgent and innovative solutions to effectively detect and provide interventions for individuals in times of need. Recent mHealth-based approaches offer promising new opportunities to address these issues through ubiquitous devices. However, the design rationale, theoretical framing, and the mechanisms through which users' perspectives and experiences guide the design and deployment of such systems have not been analyzed in any prior systematic reviews. OBJECTIVE In this paper, we systematically review these approaches and applications for their feasibility, efficacy, and usability. Further, we evaluate whether human-centered research principles and techniques guide the design and development of these systems, and examine how the current state-of-art systems apply to real-world contexts. In an effort to gauge the applicability of these systems, we also investigate whether these approaches consider the effects of stigma and the privacy concerns related to collecting data on substance use. Lastly, we examine persistent challenges in the design and large-scale adoption of substance use intervention applications and draw inspiration from other domains of mHealth to suggest actionable reforms into the design and deployment of these applications. METHODS Four databases (PubMed, IEEE, JMIR and ACM DL) were searched over a five-year period (2016 - 2021) for articles evaluating connected mHealth approaches for substance use (alcohol use, marijuana use, opioid use, tobacco use, and substance co-use). Articles that will be included describe an mHealth detection or intervention targeting substance use and provided outcomes data and a discussion of design techniques and user perspectives. Independent evaluation will be conducted by one author, followed by secondary reviewer(s) who will check and validate themes and data. RESULTS This is a protocol for a systematic review, therefore results are not yet available. We are currently in the process of selecting the studies for inclusion in the final analysis. CONCLUSIONS To the best of our knowledge, this is the first systematic review to assess real-world applicability, scalability, and use of human-centered design and evaluation techniques in mHealth approaches targeting substance use. This study is expected to identify gaps in current substance use detection and intervention mHealth technologies and inform and motivate future development of such systems.

The massive emergence of COVID-19 cases in the first phase of pandemic within an extremely short period of time suggest that an undetected earlier circulation of SARS-CoV-2 might have occurred. Given the importance of this evidence, an independent evaluation was recommended by the World Health Organization (WHO) to test a subset of samples selected on the level of positivity in ELISA assays (positive, low positive, negative) detected in our previous study of prepandemic samples collected in Italy. SARS-CoV-2 antibodies were blindly retested by two independent centers in 29 blood samples collected in the prepandemic period in Italy, 29 samples collected one year before and 11 COVID-19 control samples. The methodologies used included IgG-RBD/IgM-RBD ELISA assays, a qualitative micro-neutralization CPE-based assay, a multiplex IgG protein array, an ELISA IgM kit (Wantai), and a plaque-reduction neutralization test. The results suggest the presence of SARS-CoV-2 antibodies in some samples collected in the prepandemic period, with the oldest samples found to be positive for IgM by both laboratories collected on 10 October 2019 (Lombardy), 11 November 2019 (Lombardy) and 5 February 2020 (Lazio), the latter with neutralizing antibodies. The detection of IgM and/or IgG binding and neutralizing antibodies was strongly dependent on the different serological assays and thresholds employed, and they were not detected in control samples collected one year before. These findings, although gathered in a small and selected set of samples, highlight the importance of harmonizing serological assays for testing the spread of the SARS-CoV-2 virus and may contribute to a better understanding of future virus dynamics.

This paper addresses the field of economic measurements of the value of assets, carried out by the methods of independent expert evaluation. The mathematical principles of application, within a comparative methodical approach, of additive and multiplicative models for correcting the cost of single indicator of compared objects have been considered. The differences of mathematical basis of the compared models were analyzed. It has been shown that the ambiguity in the methodology of correction procedure requires studying the advantages and disadvantages of known models, as well as the justification and elaboration of recommendations for their application. Possible forms of correction representation using several alternative units of measurement have been defined; formulas for their interconnection have been built. Analytic expressions have been derived that mathematically describe the algorithms for performing the evaluation correction procedure using various forms of correction representation. The influence of the correction execution model on the characteristics of uncertainty in the independent evaluation result has been analyzed. The scope of two possible types of correction introduction models has been determined. A specific numerical example was used to demonstrate the methodological advantages of using a multiplicative model when summarizing percentage corrections. The independence of the correction result on the sequence of correction introduction has been confirmed. It is proposed to use the selected measure of partial corrections as a criterion for the adequacy of the correction introduction model. It is proved that the result of the independent expert evaluation depends on the chosen model and does not depend on the sequence of correction introduction. The reported study results are important in terms of theory and practice since they make it possible to improve the accuracy and reliability of the result of independent expert evaluation.

Background: Few “interventions” around suicide and stigma have reached into psychiatric institutions. Lived Lives is a science-arts approach to addressing suicide and stigma, informed by a psychobiographical and visual arts autopsy. The resulting artworks and mediated exhibition ( Lived Lives), has facilitated dialogue, response and public action around stigma-reduction, consistent with a community intervention. Recent evidence from Lived Lives moved us to consider how it may situate within a psychiatric hospital. Methods: Lived Lives manifested in St. Patrick’s University Hospital (Ireland’s oldest and largest psychiatric hospital) in November 2017.   A mixed-methods approach was used to evaluate the exhibition as a potential intervention to address stigma around suicide, with quantitative and qualitative data collected via written questionnaire and oral data collected via video documentation.  Bereavement support was available. A Clinician and an artist also provided independent evaluation. Results:  86 participants engaged with the exhibition, with 68 completing questionnaire data. Audiences included service users, policy makers, health professionals, senior hospital administrators and members of the public. 62% of participants who completed questionnaires were suicide-bereaved; 46% had experienced a mental health difficulty, and 35% had been suicidal in the past. 91% thought Lived Lives could be of benefit in the aftermath of a suicide death. Half of participants thought Lived Lives could help reduce suicidal feelings, whereas 88% thought it could benefit those with Mental Health difficulties. The emotional response was of a visceral nature, including fear, anger, sadness, disgust and anxiety. Conclusions: Lived Lives sits comfortably in discomfort, unafraid to call out the home-truths about stigma and its pervasive and pernicious impact, and with restoring identity at its core. Lived Lives can operate within a psychiatric hospital, as well as in community. The challenge is to move it forward for greater exposure and impacts in at-risk communities.

SARS-CoV-2 Antibody tests have been marketed to diagnose previous SARS-CoV-2 infection and as a test of immune status. There is a lack of evidence on the performance and clinical utility of these tests. We aimed to carry out an evaluation of 14 point of care (POC) SARS-CoV-2 antibody tests. Serum from participants with previous RT-PCR (Real-Time Polymerase chain reaction) confirmed SARS-CoV-2 infection and pre-pandemic controls were used to determine specificity and sensitivity of each POC device. Changes in sensitivity with increasing time from infection were determined on a cohort of participants. Corresponding neutralising antibody status was measured to establish whether the detection of antibodies by the POC device correlated with immune status. Paired capillary and serum samples were collected to ascertain whether POC devices performed comparably on capillary samples. Sensitivity and specificity varied between the POC devices and in general did not meet the manufacturers reported performance characteristics signifying the importance of independent evaluation of these tests. The sensitivity peaked at >20 days following symptoms onset however sensitivity of 3 POC devices evaluated at extended time points showed that sensitivity declined with time and this was particularly marked at >140 days post infection onset. This is relevant if the tests are to be used for sero-prevelence studies. Neutralising antibody data showed positive antibody results on POC devices did not necessarily confer high neutralising antibody titres and these POC devices cannot be used to determine immune status to the SARS-CoV-2 virus. Comparison of paired serum and capillary results showed that there was a decline in sensitivity using capillary blood. This has implications in the utility of the test as they are designed to be used on capillary blood by the general population.

Learning disorders can have adverse impacts on children and families extending beyond the academic skills deficits. The goal of the present study was to assess the impact of the school’s response, following an independent evaluation, for child and family school-related quality of life (QOL). We hypothesized that a positive school response would be associated with improved QOL. Parents completed the LDQOL/15, a brief measure of school-related QOL, at the time that their child received an independent evaluation and again a year later ( N =155). At follow-up, parents reported the tenor of the school’s response to the evaluation and whether special education services had changed. QOL problems were very high in this referred population, with 66% of respondents rating a level of problems in the clinical range at baseline. Predicted interactions between time of assessment (baseline/follow-up) and the school’s response to the evaluation were confirmed for all outcome variables; a positive school response was associated with improved QOL. Nevertheless, the overall prevalence of school-related QOL problems remained high a year after the evaluation. These findings indicate that special education interventions can mitigate LD impacts, but even with these services, many children with learning disorders and their families continue to be significantly affected.

Recently, the enormous advantages of cloud services make them increasingly appealing to the small and medium-sized enterprises. The growing number of available services makes it challenging to select trustworthy services. Existing approaches focus on user preferences to guide personalized services recommendation for individual users, but lack of the research on trustworthy service recommendation for the small and medium-sized enterprises that represents a group user consisting of multiple individual users. For this type of enterprise, the cloud services recommendation must address the challenges from the diverse client context of individual users, the imprecise quality of experience in an uncertain cloud environment and the invalid or unsatisfactory recommendations. A client context-aware approach is proposed to recommend trustworthy cloud services for the small and medium-sized enterprises based on non-compensatory multi-criteria decision-making. In it, a type of client context is viewed as an independent evaluation criterion, and the interval neutrosophic numbers are employed to measure the fuzzy trustworthiness of cloud services. Based on the investigated outranking relations of interval neutrosophic numbers, a non-compensatory multi-criteria decision-making procedure via an improved ELECTRE III method is developed to rank candidate services. Experimental results demonstrate that this approach could efficiently produces the accurate ranking results of cloud services and effectively recommend the trustworthy service for small and medium-sized enterprises.

AbstractThere have been few independent evaluations of computer-aided detection (CAD) software for tuberculosis (TB) screening, despite the rapidly expanding array of available CAD solutions. We developed a test library of chest X-ray (CXR) images which was blindly re-read by two TB clinicians with different levels of experience and then processed by 12 CAD software solutions. Using Xpert MTB/RIF results as the reference standard, we compared the performance characteristics of each CAD software against both an Expert and Intermediate Reader, using cut-off thresholds which were selected to match the sensitivity of each human reader. Six CAD systems performed on par with the Expert Reader (Qure.ai, DeepTek, Delft Imaging, JF Healthcare, OXIPIT, and Lunit) and one additional software (Infervision) performed on par with the Intermediate Reader only. Qure.ai, Delft Imaging and Lunit were the only software to perform significantly better than the Intermediate Reader. The majority of these CAD software showed significantly lower performance among participants with a past history of TB. The radiography equipment used to capture the CXR image was also shown to affect performance for some CAD software. TB program implementers now have a wide selection of quality CAD software solutions to utilize in their CXR screening initiatives.