Are data from national and other large scale user experience surveys used in local quality work? A systematic literature review
Mapping review
|Updated
Key message
The Norwegian Knowledge Centre for the Health Services carried out a literature review with focus on to what extent user experience data from national or other large scale surveys are being used in local quality work in the health services, together with types of follow-up of the results from such surveys.
12 publications met the inclusion criteria. Half of the included studies carried out a questionnaire survey as a follow-up after the original user experience survey. Follow-up in four of the remaining studies were qualitative interviews, one used a combination of these two methods, while one was a cluster randomized study.
Key findings in the literature review were:
- User experience data from national or other large scale surveys are being used in local quality work. There is a need for more, better and standardized research regarding the use of such data, as well as research on the interaction between contextual factors and the utilization of data.
- Information sharing is the most common type of follow-up. This could be information meetings for co-workers, meeting between quality improvement groups and the hospital board, or meetings where the results were transformed into action plans.
- That the results from user experience surveys stimulate to formal and informal discussions is another type of follow-up that is mentioned in several of the publications.
- To compare results across time and institutions are also types of follow-up that were utilized by several.
Summary
Introduction
During the spring of 2012, the Norwegian Knowledge Centre for the Health Services conducted a systematic literature review. The topic for the review was the use of user experience data in the health services. This report presents the results from the literature review.
A number of western countries have established programmes to measure user experiences with the health services. These programmes are very similar to the Norwegian system for measuring user experiences, for which the Norwegian Knowledge Centre for the Health Services is responsible. This similarity with other countries made us want to investigate international research regarding the use of data from large scale user experience surveys in local quality work. We wanted both to investigate the reported scope of utilisation of user experience data from large scale surveys, and how the results from these user experience surveys are being followed up.
The objective of this literature review is to systematically search for, identify and describe scientifically published articles about the use of user experience data from national and other large scale surveys for quality work in the health services. Both to what degree the data are being used, and types of follow-up of the results from user experience surveys are of interest. The aim is to contribute to knowledge regarding the use of large scale user experience surveys, and to enable the health services to improve user experiences by means of information from national surveys.
Method
We conducted a systematic search for literature in the following databases:
- Ovid Embase 1974-2012
- Ovid MEDLINE in-process & other non-indexed
- Ovid MEDLINE 1946-present
- Ovid PsycINFO 1967-December (week 4) 2011
In addition we conducted a search in Cochrane Database of Systematic Reviews (Cochrane Reviews). We also searched in the reference lists of relevant articles to complement the literature review. The search for studies were finished in February 2012, the search was updated August 20 th 2012.
Inclusion criteria
- Publications about user experiences and satisfaction that investigates to what extent data from national and other large scale user experience surveys are used for local quality work in the health services.
- Publications in English or a Scandinavian language.
Exclusion criteria
- Studies about use of data where the actual user experience survey is conducted with fewer than five health care providers included (hospitals, nursing homes, medical offices etc).
- Effect studies about concrete quality improvement initiatives.
- Studies where qualitative user experience data are the basis for local quality work initiatives in the health services.
Results
The search in databases identified 8 714 references. The number of references was 5 361 after deletion of duplicates and conference abstract. Search in articles’ reference lists added another seven publications. All the references were coordinated in a Reference Manager database. Out of the 5 361 unique references that were identified, 82 articles were read in full. Out of these 82, 12 were appraised as relevant based on the inclusion criteria. All 12 are primary research, and all report from a European country or the USA.
Half of the included studies carried out a questionnaire survey as the follow-up of the initial user experience survey. In these studies an estimate were made of the portion of the respondents who reported to have used the user experience data in local quality work. The proportion who reported such use varied from 54 % to 83 % in the six studies.
Regarding the six remaining studies, one reported that seven, out of eight, medical groups had stated goals for quality improvement, and that six of these had implemented interventions based on their goals. Another qualitative study reported that almost all of the respondents reported use of data to improve the service, without specifying the numbers. Two studies investigated whether the respondents regarded the results from user experience surveys as useful, to which 46 % and 23 % of the respondents replied positively. One study also asked the respondents whether they used the results often or always, to which 7 % replied that they did.
One of the included articles described an intervention where general practitioners either received the results from a user experience survey or not. The study showed that the practitioners who received such results thought of them as less relevant to utilize for changing the health care delivery compared to the practitioners who did not receive such results.
Information sharing was the most commonly cited type of follow-up in the included publications. Several of the articles report that the respondents communicate the results throughout the organization, as well as to consumers and other stakeholders. That the results from the user experience surveys stimulate to formal and informal discussions is another type of follow-up that is mentioned in several of the publications. Further, to compare results across time and institutions are also types of follow-up that are utilized by several.
Conclusion
A key finding in the literature review was that user experience data from national and other large scale surveys are being used in local quality work. However, there is a need for more, better and standardized research regarding the use of such data, as well as research on the interaction between contextual factors and the utilization of data.
Type of follow-up and the extent of this vary in the included articles. From no specific follow-up other than the follow-up survey itself, via different types of information sharing to an extensive work to assure or improve the quality are described.
It is important to assess which efforts that can increase the use of national user experience surveys. Although there is a need for more research on this topic, a considerable part of the literature in this review points to the possibility to present results on the lowest level possible (e.g. department or unit). In addition; it is important that those who are implementing quality improvement measures are receiving information and training in improvement methods and improvement efforts.