Hopp til innhold

Get alerts of updates about «Implementation of machine learning in evidence syntheses in the Cluster for Reviews and Health Technology Assessments: Final report 2020-2021»

How often would you like to receive alerts from fhi.no? (This affects all your alerts)
Do you also want alerts about:

The email address you register will only be used to send you these alerts. You can cancel your alerts and delete your email address at any time by following the link in the alerts you receive.
Read more about the privacy policy for fhi.no

You have subscribed to alerts about:

  • Implementation of machine learning in evidence syntheses in the Cluster for Reviews and Health Technology Assessments: Final report 2020-2021

Report

Implementation of machine learning in evidence syntheses in the Cluster for Reviews and Health Technology Assessments: Final report 2020-2021

Published

We tested and documented pros and cons of using Machine Learning in various phases of the conduct of various evidence syntheses.

Forside Sluttrapport.jpg

We tested and documented pros and cons of using Machine Learning in various phases of the conduct of various evidence syntheses.


Downloadable as PDF. In English. Norwegian key messages.

About this publication

  • Year: 2021
  • By: Norwegian Institute of Public Health
  • Authors Muller EA, Ames H, Himmels J, Jardim PJ, Nguyen L, Rose C, Ven de Velde S.
  • ISBN (digital): 978-82-8406-231-0

Key message

Machine learning (ML) has the potential to increase the efficiency of evidence syntheses. During 2020-2021, a team in the division for Health Services at the Norwegian Institute of Public Health, tested and documented pros and cons of using ML in various phases of the conduct of various evidence syntheses, and built employees’ competence in using ML. This report describes the work undertaken by the ML team, project results and lessons learned.

The ML team focused attention on ML functions and systems available within EPPI Reviewer: Priority screening, Custom and Pre-built classifiers, RobotReviewer to assess Risk of Bias, Automatic text clustering, and Microsoft Academic Graph (MAG). We implemented ML functions across 19 project teams and trained 23 employees. We found that utilizing ML in our reviews increased speed, with no identified threats to methodological quality. Screening time was reduced by 60-90% in all projects. Automated study categorization – while applicable to a smaller range of projects – reduced manual time in this phase by 60-70%. 

ML can, and should, change usual project workflows. The review process can become less linear and more cyclical, and several tasks can be conducted in parallel. However, workflow changes are not insignificant for those involved, and future ML work would benefit from a structured approach to both change management and innovation diffusion.

The report concludes with lessons learned and experiences gained. They shaped our proposals for future ML strategies, covering capacity-building, innovative activities, evaluation of effect, and workflow optimization.

This report was first published on August 11th, 2021.

The PDF-file was removed  17th of August 2021, and the text 

«This report is temporarily removed for editing of e.g. preface and executive summary. The updated report will be made available in early September” was added.

A new PDF-file was published on september the third, 2021.