Implementation of Machine Learning in Division for Health Services: Strategy Proposal from 2024
Report
|Published
This report presents Machine Learning 3.0s strategy suggestion from 2024 and onwards based on the team’s experiences from its inception in 2020.
Key message
In 2020, the Cluster for Reviews and Health Technology Assessments (HTV) at the Norwegian Institute of Public Health (NIPH) established a dedicated machine learning (ML) team. The ML team has since become an international leader in integrating and implementing ML into evidence synthesis.
The overall goal of the ML team is to use ML in a way that best combines human intelligence and ML, to enhance human activities, by figuring out how best to integrate ML and workflow changes, throughout the review process.
This report presents ML 3.0s strategy suggestion from 2024 and onwards based on the team’s experiences from its inception in 2020.
In response to the evolving needs of our institute and in alignment with the strategic objectives of both the institute and the division, this proposal advocates for the elevation of the ML team to division level from 2024 and onwards. This will ensure long-term sustainability and decrease financial burdens for HTV. In addition, we propose a restructured organizational framework with three teams: Innovation and Horizon Scanning, Evaluation and Evidence Building, and Implementation and Support, as well as a steering committee to coordinate activities and engage in external networking.
Summary
Background
In 2020, the Cluster for Reviews and Health Technology Assessments (HTV) at the Norwegian Institute of Public Health (NIPH) established a dedicated machine learning (ML) team, aligning with NIPH's strategies for automation and workflow innovation. This was also in line with the goals of the Division for Health Services which sought to automate work processes and efficiently summarize evidence using ML. Since its inception, the ML team has positioned NIPH as a leader in implementing ML into evidence synthesis, achieving significant milestones, and securing official financing in November 2022.
The report presents suggestions for the strategy from 2024 and onwards based on the experiences of ML implementation from 2020 up to the latest iteration of the team, ML 3.0.
Suggested short-term strategy and focus areas
The proposed short-term strategy aims to address the imminent challenges faced by the ML team within HTV, particularly the departure of key members. The focus is twofold: first, to keep the ML team operational throughout 2024, and second, to prioritize activities aligning with HTV's overarching ML goals. The activity plan includes recruiting a new leader with ML experience and 1-2 team members, including at least one information specialist. The short-term strategy centres on maintaining and enhancing current employees' ML knowledge. Capacity-building activities will be guided by employees' preferences, including peer-to-peer support, technical workshops by function experts, and expanding the successful e-learning initiative from 2023 to cover additional ML aspects. This approach aims to ensure continuity, upskilling, and sustained focus on ML goals in the short term.
Suggested long-term ML strategy
In response to the evolving needs of our institute and in alignment with strategic objectives of both the institute and the division, this proposal advocates for the elevation of the ML team from cluster level to division level as a long-term strategy. The proposal aligns with both the institute and division strategies, emphasizing the importance of evolving infrastructure, knowledge support, and expertise in innovative methods like ML. Elevating the ML team is seen as imperative for innovation and collaboration in evidence synthesis and public health, preventing limitations in capacity and demotivation of team members.
Elevation of the team to division level will benefit HTVs work with evidence syntheses as it will ensure long-term sustainability of our ML efforts, more freedom in exploring ML tools aimed at specific evidence synthesis processes, as well as decreasing financial burdens for HTV.
Without this transition, there is an elevated risk of limited capacity, tasks, and time for the team, hindering its ability to keep up with advancements in the field. There is also the potential of becoming merely a “maintenance team,” leading to demotivation and high turnover.
ML Team 4.0 organization
To address the challenges of rapid growth in ML and artificial intelligence (AI) tools, a restructured organizational framework for the ML efforts is proposed, dividing the team into three distinct teams: Innovation and Horizon Scanning, Evaluation and Evidence Building, and Implementation and Support. Within each team we suggest a further two-pronged division of activities: One related to the use of ML and AI in evidence synthesis and one related to the use of ML and AI in other research areas, like primary research, registries etc. This will ensure a continued investment and development of ML in the evidence synthesis process, as well as acknowledging the increased focus on primary research and registry work within the division and NIPH as a whole. Each team would have specific responsibilities, and a steering committee comprising the team leads would be established to coordinate activities and engage in external networking. The restructuring aims to enhance resource efficiency, introduce new competencies, increase the talent pool of potential new members, and facilitate collaboration across departments. Clear criteria for team members are outlined based on their roles, emphasizing skills in ML and AI, communication, teaching, and experience in evaluation and implementation.
Suggested focus areas for long-term strategy
The current iteration of the ML team has put forward suggestions for key focus areas for the next iteration of the team, which align with the overarching goals of the division and institute. These include seeking external funding for financial sustainability, continue exploring the use of OpenAlex in evidence synthesis in collaboration with the librarian team, and fostering continued collaboration with external institutions, particularly EPPI Centre. The team also aims to strengthen interdisciplinary collaboration within NIPH, build expertise in Generative AI, develop a comprehensive ML implementation package for training other institutions, explore and evaluate data extraction tools, and investigate the application of ML/AI in the institute's registry work.
