Cost and Quality in Health Care

RESEARCH ARTICLE Open Access

Exploring resistance to implementation of welfare technology in municipal healthcare services – a longitudinal case study Etty R. Nilsen1*, Janne Dugstad2, Hilde Eide2, Monika Knudsen Gullslett2 and Tom Eide2

Having Trouble Meeting Your Deadline?

Get your assignment on Cost and Quality in Health Care completed on time. avoid delay and – ORDER NOW

Get a 20% Discount

Abstract

Background: Industrialized and welfare societies are faced with vast challenges in the field of healthcare in the years to come. New technological opportunities and implementation of welfare technology through co-creation are considered part of the solution to this challenge. Resistance to new technology and resistance to change is, however, assumed to rise from employees, care receivers and next of kin. The purpose of this article is to identify and describe forms of resistance that emerged in five municipalities during a technology implementation project as part of the care for older people.

Methods: This is a longitudinal, single-embedded case study with elements of action research, following an implementation of welfare technology in the municipal healthcare services. Participants included staff from the municipalities, a network of technology developers and a group of researchers. Data from interviews, focus groups and participatory observation were analysed.

Results: Resistance to co-creation and implementation was found in all groups of stakeholders, mirroring the complexity of the municipal context. Four main forms of resistance were identified: 1) organizational resistance, 2) cultural resistance, 3) technological resistance and 4) ethical resistance, each including several subforms. The resistance emerges from a variety of perceived threats, partly parallel to, partly across the four main forms of resistance, such as a) threats to stability and predictability (fear of change), b) threats to role and group identity (fear of losing power or control) and c) threats to basic healthcare values (fear of losing moral or professional integrity).

Conclusion: The study refines the categorization of resistance to the implementation of welfare technology in healthcare settings. It identifies resistance categories, how resistance changes over time and suggests that resistance may play a productive role when the implementation is organized as a co-creation process. This indicates that the importance of organizational translation between professional cultures should not be underestimated, and supports research indicating that focus on co-initiation in the initial phase of implementation projects may help prevent different forms of resistance in complex co-creation processes.

Keywords: Ethical resistance, Welfare technology, Innovation, Co-creation, Municipal healthcare

* Correspondence: [email protected] 1The Science Centre Health and Technology, School of Business, University College of Southeast Norway, Postboks 235, N-3603 Kongsberg, Norway Full list of author information is available at the end of the article

© The Author(s). 2016 Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.

Nilsen et al. BMC Health Services Research (2016) 16:657 DOI 10.1186/s12913-016-1913-5

Background Healthcare services face vast challenges that will increase in the years to come, partly due to demographic changes including ageing populations [1, 2]. Welfare technology is viewed as one important means to meet these chal- lenges. Implementation of digital night surveillance tech- nologies in nursing homes and home care services has emerged as a potentially efficient way of meeting the need for monitoring persons for healthcare and safety reasons. This is an alternative to calling in on, for example, patients with dementia or intellectual disabilities, and potentially waking them up at night. However, the application and use of digital surveillance technologies in the care for vul- nerable individuals generates considerable ethical debate [3–5]. Implementation of welfare technology also implies innovation and organizational change, which is often met by different kinds of resistance. Resistance can be found on individual, organizational, and institutional levels, and these levels are often inter-connected [6–8]. This paper explores if and how resistance occurs on different levels in the initial phase of digital surveillance technology imple- mentation in municipal nursing homes and home care services.

Implementation of innovation Innovation has been defined as “the intentional introduc- tion and application within a role, group, or organization, of ideas, processes, products or procedures, new to the relevant unit of adoption, designed to significantly benefit the individual, the group, or wider society” [9, 10]. This definition has become widely accepted among researchers [11, 12]. It captures many aspects of the innovation process under study, as it aims at implementing new tech- nologies and developing new ways of working in order to benefit the individual service user and the healthcare organization. Implementation is seen as one of the four stages of innovation: dissemination, adoption, implemen- tation and continuation [13]. The implementation stage is according to Rogers “that which occurs when an individ- ual puts an innovation into use” ([14]:474). Implementation of technology initiates a change process

and has the potential to alter the way we work, how we organize work and the power relations in an organization. However, a large number of change initiatives fail due to unfocused and insecure management and lack of systematic project management [15, 16] or are slow to be implemented (e.g. [17–19]). The implementation phase is increasingly becoming a phase where the technology developers and the customers cooperate closely, and in the business literature it is coined as co-development of the product [20] or co-creation of value [21]. The concept of co-creation implies close and continuous interaction in the implementation phase between the innovators and developers of the technology and the customers.

The technology developers may lack knowledge about the market and the users, while customers often also lack familiarity of technological language and technol- ogy proficiency. In the implementation phase of, for ex- ample, welfare technology, several knowledge spheres or epistemic cultures meet [22].

Resistance to technology implementation Resistance is inherent to organizational life [23, 24], and the literature on resistance stretches across several disci- plines [25]. According to a recent review of research on resistance to healthcare information technologies, resist- ance is under-researched and multifaceted, and relatively little attention has been paid in understanding it [26]. Resistance to change has mainly been seen as an effort to maintain status quo and research has traditionally seen resistance as a negative force that must be overcome [23], and as a restraining force “that leads employees away from supporting changes proposed by managers” [27:784]. Resistance to technology implementation is ‘expected’ and can be seen as the flip side of success factors for innovation which has been emphasized in research on technology implementation in the Information Systems (IS) field (see for instance [26, 28]). Change processes like the implementation of technology

are met by several types of resistance. Resistance is found at individual, organizational and institutional levels [6–8], and these levels are inter-connected. Previous research has for instance shown that traditional organizational constel- lations may change as a result of technology implementa- tion [29, 30]. Increased use of technology may change the work pattern, the division of labour and the interaction pattern. Previous research also indicates that the imple- mentation is complicated by a lack of training and lack of interest from employees [31, 32]. Within the IS field, research on resistance concentrates

on the negative paradigm, focusing on subordinates' un- willingness to implement decisions made by the manage- ment [33, 34]. Resistance occurs if threats are perceived from the interaction between the object of resistance and initial conditions [33]. Resistance creates friction, which has negative connotations and may complicate the imple- mentation process. Friction is however also an antecedent to change [35]. As the implementation process proceeds, the users are likely to make moderations to the set of initial conditions or the subject of resistance, based on their experience with the technology. Hence the nature of the resistance will change through the implementation process [33], and resistance is not considered as purely harmful. A further example is the notion of productive re- sistance [23]. Productive resistance builds on the notion of resistance as a way of coproducing change and “refers to those forms of protest that develop outside of institutional channels” [23:801].

Nilsen et al. BMC Health Services Research (2016) 16:657 Page 2 of 14

In this study, we investigate how resisters think, how they understand their own resistance and what resisters do “rather than seeing resistance as fixed opposition between irreconcilable adversaries” [23:801]. This re- sistance behaviour is categorized by Coetsee [36] as ap- athy, passive resistance, active resistance and aggressive resistance.

Resistance to technology implementation in healthcare Resistance to increased use of technology in healthcare is still considered to be under-researched [26, 29]. Lluch states in a review article on health information technolo- gies (HIT) that “more information is needed regarding organizational change, incentives, liability issues, end-users’ HIT competences and skills, structure and work process is- sues involved in realizing the benefits from HIT” [31:849]. Furthermore, the healthcare field is not one field, and

healthcare technology consists of a wide range of technol- ogy. Within the healthcare field, hospitals have often been the preferred empirical setting (see for example [33, 37, 38]), and physicians are the preferred actors under study (see for example [18, 37]). The municipal healthcare set- ting differs from that of a hospital, especially due to the organizational and structural elements of the municipality itself. The municipality is more complex and consists of several organizations, weakly tied and embedded in the larger municipal organization. Still, the levels and the various actors and units within the greater municipal organization are linked through the tasks and the users of the services. Further, the focus on patients’ interests in healthcare in general and concerning the increased use of technology, in particular, has led to focus on the groups who need to collaborate in order to implement technology [39]. Based on their studies of the implementation of infor-

mation technology (IT) in hospital settings, Lapointe and Rivard [33] identified five basic components of resistance: Resistance behaviours (from passive uncooperative to ag- gressive), the object of resistance (the content of what is being resisted), perceived threats (negative consequences that are expected implications of the change), initial conditions (such as established distributions of power or established routines) and finally the subject of resistance (the entity, individual or group, that adopts resistance be- haviours). They propose a dynamic explanation for resist- ance to the implementation of technology. The resistance behaviours result from the nature of perceived threats on various points in the implementation process. Depending on what triggers the resistance behaviours, new threats and consequently, new resistance behaviour emerges. The perceived threats and the resistance behaviour can be found at an individual and group level. In this article, we recognize the five basic components of resistance identified by Lapointe and Rivard, and define resistance

descriptively as behaviours (attitudes, acts and omissions) that obstruct or interfere with the process of co-creation and organizational change.

The case of Digital Night Surveillance The innovation project at hand is called “Digital Night Surveillance”, which is a government funded project where five municipalities, both rural and urban, work with a net- work of technology developers to develop and implement the use of sensors and digital communication in nursing homes and home care services. The project entailed service development and technol-

ogy development in a co-creation process [21, 40] within a triple-helix inspired network [41], consisting of (1) a net- work of small- and medium-size technology enterprises (SMEs), (2) municipal health and care services, and (3) a university research group [42]. The overall aim was to de- velop and implement the best possible solution to the challenges of night surveillance, in order to enhance se- curity and quality of care for the service users within the municipalities’ limited resources [29, 43]. The co-creation and implementation process was facilitated by a profes- sional manager or “orchestrator” [42]. The technology to be implemented included sensors

on doors and in electronic security blankets (on mat- tresses) used during the night. A web-based portal facili- tated communication via traditional PCs as well as mobile devices, such as tablet computers and smartphones. Most of the municipal services already had some welfare tech- nology installed, such as alarm systems. The novelty of the new system was tied to the web-based portal into which different technological applications could be connected and administered. In this way, technology in different cat- egories and from different producers could function to- gether and be programmed and adjusted to the individual patients’ needs. Alterations could be made based on for instance variations in needs during the day or due to the progression of a disease. An alarm went off when an incident happened. The system was programmed to send alarm messages to dedicated personnel, and they received the alarm on either a smartphone, pad or PC, or a combination of these. They ‘signed out’ the alarm as they checked on the patient. The implementation project involved a large number

of stakeholders, and the study of resistance involved ex- ploring some of these. Data in this study comes mainly from the healthcare providers on the night shift, managers on various levels in the municipalities and healthcare insti- tutions, and the technology developers, who also installed the equipment and trained the healthcare providers. Furthermore, the following stakeholders were involved and/or affected by the project: IT service staff, patients and families.

Nilsen et al. BMC Health Services Research (2016) 16:657 Page 3 of 14

The home care services and the nursing homes in- cluded in the project had primary users in need of night supervision. The residents of the nursing homes suffered from dementia, and tended to get up at night and wan- der around, which has been described as one of the most challenging behaviours to manage [44]. Night surveil- lance in one form or another (face-to-face or technology based) was necessary to detect “night wanderers” and guide them back to bed in order to avoid confusion and anxiety, avoid the risk of falling and injuries, and protect other residents from being disturbed and frightened at night. In the Digital Night Surveillance project, sensors in blankets and on doors detected and sent a signal if the pa- tient left the room. The patients did not actively use the technology; rather the users were the healthcare providers. The participating municipalities identified a need for

innovation in order to ensure safety at night for the ser- vice users. Then entered into a contract with a network orchestrator, a network of technological SMEs and a sci- ence centre for health and technology in a university, in order to run an implementation project, which included both municipal home care services and nursing homes. The initiative came from the empirical field itself.

Methods Aim and study design The aim of this study was to explore resistance to imple- mentation of welfare technology in five municipalities in

Norway. The design was explorative and draws on a lon- gitudinal single-embedded case study [45] with elements of action research. The study was carried out during 2013 and 2014. A case study is suitable for an explorative, in-depth

study of contemporary events in its real-life context [45]. The case was a project, organized with sub-projects in each of the municipalities, with a local project manager on site. The research took a multi-stakeholder perspective as both the technology developers in the business net- work, who also install the technology and train the health- care providers, and the healthcare providers, on various levels of the homecare services and nursing homes, were included in the study. The healthcare providers are the ac- tual users of the technology and are defined as the users in our study. The study does not include data from the end-users. Three main action research elements were applied: 1)

researcher participation in the project design and planning activities, 2) researcher participation in (and by occasion also facilitation of) knowledge sharing and reflection processes during workshops and meetings, including presentation of preliminary research findings, and 3) using focus group interviews not only to collect data but also to stimulate critical reflection on the co-creation and imple- mentation process [46, 47]. Table 1 gives an overview of the longitudinal design,

the timeline, the technology, the users and the data col- lection methods.

Table 1 Design and data collection methods – an overview

Stake-holders Technology Research activities

Q3 2013 Q4 2013 Q1 2014 Q2 2014

Municipality 1 Sensor technology Alarm system Web-based portal Installations: 8

EP WS PO FG

WS PO II

WS PO

WS PO

Municipality 2 Sensor technology Alarm system Web-based portal Installations: 11

EP WS PO FG

WS PO II

WS II PO

WS PO

Municipality 3 Sensor technology Alarm system Web-based portal Installations: 9

EP WS PO FG

WS PO II

WS PO

WS PO

Municipality 4 Sensor technology Alarm system Web-based portal Installations: 4

EP WS PO

WS PO

WS PO

Municipality 5 Sensor technology Alarm system Web-based portal Installations: 2

EP WS PO

WS PO

WS PO

Suppliers FG WS

WS FG WS

WS

Participants in each workshop 24 33 17 32

Abbreviations: EP Entered the project, II Individual interviews; FG Focus group interviews; PO: Participatory observation; WS Workshops

Nilsen et al. BMC Health Services Research (2016) 16:657 Page 4 of 14

Data collected The main sources of qualitative data were semi-structured interviews, both individual and focus group interviews, and observations in workshops and meetings. Altogether, data were collected through nine individual interviews, three focus group interviews and observations on site and in four workshops. In all, about 50 individuals (including the five researchers) took part in the workshops and meet- ings. The researchers facilitated some of the workshops in order to stimulate co-creation and the production of process data. Twenty-one individuals were interviewed, both healthcare providers (from all five municipalities) and technology developers. All interviewed informants participated in two or more of the workshops. Some of the participants in the focus groups were also interviewed in-depth individually. All participants consented to partici- pation in the research study. The selection of informants from the municipalities

for the individual interviews was aided by the project managers. The inclusion criteria were employees work- ing as either project manager, middle manager or night healthcare provider. Eight women and one man were interviewed in the period from September 2013 to November 2014. Four technology developers, all male, participated in a focus group interview in January 2014. The focus group method was in line with the methodology used in the project itself, which used the workshops as an arena for orchestrated interaction, collective reflection, knowledge sharing and innovation of services [42], thereby the interviews were an arena for co-creation in themselves [48]. The in-depth interviews followed a semi-structured interview guide (Additional file 1) [49, 50] and were car- ried out as conversations. An interview guide was used as a checklist at the end of the interview to ensure that all planned topics were included. The first two focus group interviews with healthcare providers from three of the mu- nicipalities were performed as part of a workshop ar- ranged early in the implementation phase, and were conducted by four of the researchers. The third focus group interview was conducted by two of the researchers with central representatives from the network of technol- ogy companies. The focus group interviews were con- ducted face-to-face and lasted for about 90 min each. Both the in-depth interviews and the focus group interview were digitally recorded and transcribed verbatim.

Data analysis Data from the interviews and observations were analysed and interpreted as inspired by Kvale’s description of the bricolage approach to data analysis [49]. Analysing data based on bricolage involves the use of various tech- niques and concepts during the process. We also used researcher triangulation [51], which meant that the whole research team with members from various fields such as

organization and innovation studies, sociology, psych- ology, nursing, healthcare research and ethics, took part in the analysis and interpretation process. The main reason for choosing a researcher triangulation approach was the need for different perspectives to understand the complexity of the innovation and co-creation process, involving five different municipalities, including differ- ent professional roles, service designs, IT systems, and local decision-making procedures. As a first step, following the description of analysis by

Kvale and Brinkmann [49], the transcribed texts from the interviews were systematically read through in a naïve manner. A reflexive, open-minded and inductive reading was pursued, as well as grasping the intuitive meaning of the text as a whole and to interpret the participants’ ex- perience and descriptions of the implementation of wel- fare technology. The themes in the analysis arose in an iterative process between reading and interpreting by sev- eral researchers, in order to find meaningful units and then themes according to the research question [49, 52]. Threats to validity were met by cooperating within the

research team in all phases of the research project, which ensured an open discussion as well as deep knowledge of the context. The reliability of the study was strengthened through researcher triangulation and continuous contact with the network. Threats to reliability have further been met by describing the research approach in detail.

Results At the outset, there were few signs of resistance among the participants. As the process moved on, various forms of resistant behaviour emerged, from scepticism of the usefulness and the functionality and safety of the tech- nology, to both passive and more active uncooperative attitudes towards the change of initial conditions, such as established routines, practices and technological infra- structure. The perceived threats were often communicated indirectly, and not always easy to identify, but in many cases, they were associated with technological instability, feelings of uncertainty and concerns for the quality of care. Resistance was found in different groups of participants and on different levels of the municipal organization. Four categories of resistance with several subcategories were identified, as laid out in Table 2. In the following, the findings will be presented in more

detail and exemplified, starting with organizational issues.

Organizational resistance Resistance to change in established routines The surveillance technology was primarily introduced on the night shift, and only the night shift personnel were trained to use it. Usually, the employees worked ei- ther only night shifts or only day/evening shifts, and there was only brief contact between the shifts. The use of the

Nilsen et al. BMC Health Services Research (2016) 16:657 Page 5 of 14

technology appeared to demand a closer cooperation be- tween the shifts. For instance, there was a need for the evening shift to prepare the technology while the patients were still awake. A night shift worker said: “We need to have good cooperation with them, so that the mattresses are placed correctly in the evening and that they are switched on the way they are supposed to.” Another night shift worker put it this way:

The day shift must make sure that things work, do things well, so that I can do a good job. I cannot ask the patients to wake up and get out of bed so that I can check that everything is OK in bed. That would be stupid.

The needs for adjusted routines and better communi- cation and cooperation between day/evening and night shifts were soon recognized. However, both project man- agers and healthcare personnel experienced a lack of interest and support from the responsible middle man- agers and unit leaders or ward nurses. As one of the pro- ject managers answered when asked whether the unit leader had taken an active role in the project: “No, she has barely participated and does not take the role. And she feels it is fine that I have that role”. This lack of managerial interest and omission to make

the necessary adjustments to established routines (which was beyond the authority of the project leaders) may be interpreted as a passive form of organizational resistance to change, which interfered with, and to some degree obstructed, the process of co-creation and implementation.

Resistance to necessary competence building The day shift did not receive any training in how to pre- pare and use the technology, and would hear about the project only through information in staff meetings. The need for training of the day shift personnel was soon

recognized by the project leaders and the other partici- pants, but the responsible unit leaders did not arrange for such. The lack of interest from the management in competence building across shifts resulted in a poor un- derstanding of the project and the technology on the part of the day shift. One of the personnel working night shift declared:

I feel that they do not understand any of this. It is a «night-shift-thing». (…) and I do not think they follow up, because it is never talked about. So I hoped we could have a more thorough conversation about this, not just two minutes in the staff meeting.

Systemic resistance to communication across groups and professions In addition to the lack of communication and cooperation between shifts, a more general issue emerged concerning communication, knowledge transfer and organizational learning. Communication channels across organizational levels, units and groups of professions within the complex municipal system were scarce. Those involved in the implementation of the surveillance technology lacked sufficient information about, for example, potential risks. Accordingly, this was an issue in workshops and inter-municipal meetings. However, not everybody in- volved could attend the workshops, and some groups – such as the cleaning staff – were not thought of as having a role in the implementation process. An example of an unforeseen risk, which proved to be a problem, was that cleaning personnel – not being sufficiently informed – on occasions moved electronic plugs and equipment in order to clean behind desks and in the corners. Breaking the electrical circuit might have the effect that sensors or communication devices shut down, and the error had to be detected before the system could be made functional again. The lack of communication channels across groups, levels and professions may represent an organizational re- sistance that made it difficult to prepare for unexpected errors that might obstruct or interfere with a successful implementation and use. During the workshops, it became clear that the procedures and written instructions had to include more groups than initially thought of.

Management resistance to participatory processes Little by little it became clear that neither the steering group nor the responsible municipal leaders or their central IT support departments had taken sufficient measures to ensure that the necessary infrastructure was in place to serve the participating homecare units and nursing homes. It appeared that the municipalities’ IT support departments had not been included in the initial phase of the project. This was in spite of the well-known fact that the innovation technology in question required

Table 2 Categories of resistance

Main categories Subcategories

Organizational resistance

• Resistance to change in established routines • Resistance to necessary competence building • Systemic resistance to communication across groups and professions

• Management resistance to participatory processes

Cultural resistance • Resistance due to language differences • Resistance due to a clash of professional cultures • Resistance against the role as co-creator

Technological resistance

• Healthcare providers’ resistance to technology • Resistance represented by IT infrastructure • IT support staff’s resistance to innovative practice

Ethical resistance • Resistance due to patient safety issues • Resistance due to concern for the quality of care • Resistance due to patient privacy and dignity issues • Resistance due to issues of justice

Nilsen et al. BMC Health Services Research (2016) 16:657 Page 6 of 14

a stable technological infrastructure in order to work. If the IT support department was included, this happened at a late stage in the planning process or in the imple- mentation process itself. Since the initiation of the im- plementation usually was run on the administrative level, and the crucial role of the municipal IT infra- structure would have been easy to foresee, the omission to involve the IT departments may be interpreted as a passive form of leadership resistance to collaborative and participatory processes, putting the project at con- siderable risk.

Cultural resistance The nature of the implementation project required close collaboration and interaction between different groups coming from different organizational cultures, such as the technology developers, the healthcare providers and the municipal IT staff. This collaboration was a field for learning for all parties, but also a source of resistance, that challenged established in-crowd language, profes- sional roles, administrative routines, distribution of power and decision-making responsibilities.

Resistance due to language differences There was a noticeable difference in vocabulary between the technology developers and the healthcare personnel. One healthcare provider put it this way: “I feel they miss out on the language that they use – or what do you call it? Terminology?”. The language gap was recognized also by the technology developers, but hard to bridge. One of them explained it as a question of awareness:

We still have a tendency to use words and concepts from our world that we use on a daily basis, that we are actually not aware of that we use, but we can see that their eyes become glassy. And if they do not understand, they do not say so. It is a challenge.

Resistance due to a clash of professional cultures Communication problems between the technology de- velopers and the healthcare providers went deeper than language only. Trained in different professional fields and focusing delivery of very different services (techno- logical solutions vs care for vulnerable people), the cultural differences were considerable. This was observed during the first workshops. Both groups often used us–them language when speaking about each other, and initially there was some resistance on both sides to take the perspective of the other and actively enter into cooper- ation. An example is the technology developers’ reluc- tance to meet the healthcare providers’ needs for more written material on the technological procedures. This was clearly communicated from the outset, without being recognized. Instead, the developers adopted a passive

uncooperative attitude, omitting to create the material needed. As one of the technology developers expressed: “At the outset we hardly had any material at all. Be- cause we perceived that this was intuitive and straightforward”.

Resistance against the role as co-creator Like the technology developers, it took a while before the healthcare providers understood their role as co-creators. The imperfections of the technology were a constant source of concern to them. For instance, alarms would go off when they should not, and vice versa. Most healthcare providers considered technological errors to be the devel- opers’ problem, not a shared responsibility. Co-creation was perceived as foreign to them and to some degree also as a threat to their professional identity. However, some providers tried to encourage cooperation and to bridge the gap between them and the developers:

It is a pilot project, and as I said to NN [technology developer], everyone has not understood that. That we should not have a negative attitude towards everything that we are testing out. We can be negative when the project is over, if nothing works.

This clash of professional cultures was to some degree anticipated by the orchestrator, designing the workshops partly with the aim of two-way cultural translation and learning. It was a steep learning curve for both parties. The technology developers learned a lot about healthcare and started using some of the healthcare vocabulary. Like- wise, the healthcare providers became more familiar with the technology and the developers’ way of thinking: “When I am with them now I understand more what they mean and what they are talking about, because I am more into the system…” The communication and mutual understanding im-

proved in the course of the project. New material was developed, the vocabulary changed, more procedures were included, and material was also customized to each municipality and to different groups of users (healthcare personnel, patients and relatives). However, this was pri- marily done by the local municipal project managers. They had expected the technologists to take more re- sponsibility for improving and customizing the material. From their point of view, elements of passive resistance behaviours among the developers did not diminish.

Technological resistance Under the heading “Technological resistance”, we group both the resistance to the technology and the resistance represented by the technology itself.

Nilsen et al. BMC Health Services Research (2016) 16:657 Page 7 of 14

Healthcare providers’ resistance to technology To some of the healthcare providers the technology was in itself threatening. It challenged their sense of predict- ability, professionality and competence, which influenced their motivation to use the technology negatively. A main source of resistance was fear of not coping with the new technology. To some this was due to lack of familiarity with sensor technology and/or digital communication devices, and to others due to negative experiences with technology in the past. An example of the latter was a healthcare provider who for weeks had dreaded partici- pation in a training session and even considered asking for a sick leave. She remembered her negative experience with the implementation of electronic patient records some years prior, when she ended up with a frozen shoulder. As the healthcare providers’ experience with the technology and the understanding of its prospects increased, however, the resistance decreased and the at- titude became increasingly positive and enthusiastic. One of them expressed it this way: “On our team, we have a positive attitude towards this. I believe many of them find this exciting.”

Resistance represented by the IT infrastructure Perhaps the most resistant subject of resistance, interfer- ing with and to some degree obstructing a successful co- creation and implementation process, was the municipal IT infrastructure itself. In several of the municipalities, the technological infrastructure was in its infancy, and in some institutions, internet was not installed. If it was installed, it was often very unstable. As one of the healthcare providers said:

And the fact that our network is down a lot, and the system in the whole municipality is very difficult to handle, as NN [technology developer] and they have said, it is very hard to handle. And that has made it very difficult for the technology developers and us. Well, it did not matter that much for us, but as the project was going to be terminated soon they needed to have it running, and it was very difficult. I did feel a bit sorry for them.

The technology developers described it like this:

We knew that there were differences, but when you really get out there you see how it works and a lot of things fall in place. And there are large differences in the infrastructure, some places they do not have a network at all, and do not use it for anything, no technology. Other places they use a lot.

According to both the healthcare providers and the technology developers, the technological platform and

the infrastructure did not provide the necessary stability for digital surveillance at night.

IT support staff’s resistance to innovative practice The co-creation and implementation of technology in the making also required close cooperation with the central IT department and the support staff in the municipalities. The developers could experience resistance from the support staff in the form of reluctance or sometimes uncooperative attitudes, making implementation difficult. The developers themselves explained this by pointing to a contradiction in logics between the IT support whose focus was an efficient system maintenance, safety and predictability, and an innovative practice, implying co- creation and implementation of new technology:

From a technological point of view, it is very difficult to innovate in a sector that… where there is a contradiction between running efficiently and innovation. Because… IT in the municipalities have stability as their main goal, and innovation leads to instability, at least when you want to try out brand new technology.

One example of resistance to innovative practice was the reluctance to change established IT system routines. In most of the municipalities, there were routines for running the system updates during the night. This is in- compatible with the use of digital night surveillance within the same system, because it represents a threat to the security of the patients when the system is shut down in order to run updates. The healthcare providers became aware of this routine only after they started using the new technology. A healthcare provider explained how this routine interfered with successful digital night surveillance:

They run updates once a week, and at that time we cannot register and write reports. And when I entered to turn off the alarms, the system was down. So I could not get them turned off, so they just continued to go off. And all that was hopeless. And then my whole tool [technology] is wasted. And time and again they ran the updates during the night.

Ethical resistance From the very beginning, healthcare providers, even in- dividuals with a generally positive attitude towards tech- nology and innovation, expressed moral concerns. One such concern was whether the motivation behind the project was morally good or not, if it was initiated in order to enhance the quality of care or to lower the cost. “I find it [welfare technology] the right way to go. But the ethical part of it, that I'm concerned with. Not to do it to

Nilsen et al. BMC Health Services Research (2016) 16:657 Page 8 of 14

save money. That would be quite wrong.” The implicitly perceived threat seemed to be an imagined future where implementation of welfare technology is a means of budget control at the expense of competent healthcare.

Resistance due to patient safety issues Resistance among healthcare providers emerged also from a concern for patient safety and from fearing that the implementation of an unstable surveillance technol- ogy might cause adverse events and harm to patients. As the stability of the technology increased, however, this attitude of scepticism and resistance changed during the project period. A member of the staff put it this way:

Thus, it [the technology] really makes the night shift feel safe. You can just watch the smartphones and see that the patient is sleeping, and we have had on-call staff at night who were very impressed.

Resistance due to concern for the quality of care Concern for the quality of care was evident from the start. Some perceived the surveillance technology as a threat to preconditions for maintaining a high professional standard, like face-to-face communication, attentive ob- servation, tacit knowledge and professional judgement. When, for example, the healthcare providers no longer needed to enter the patient’s room at night unless the alarm on her smartphone went off, she felt like she was missing important information that she would have got if she had been physically present in the room. This included smelling and seeing the whole picture and, at times, communicating with the patient. As one informant expressed it:

but there is something about, as I am saying, when I enter a patient room then there is something about what I see and smell and find out how things are as a whole, plus he [the patient] might say that today I would like to watch TV a bit longer… for example.

Resistance due to patient privacy and dignity issues There was also a concern for patient privacy and dignity and how this would be ensured. Was not surveillance an invasion of patient privacy, and a threat to privacy at work? These questions were subject to moral delibera- tions from the start:

I have no problem displaying what I do at work. I rather think of the user, of … Where did the privacy go? I enter and leave the room and do my job, and am supposed to be professional. But the users shall feel that they have a private life when they enter their flat, that they are not going to be under surveillance, 'cause that is unnatural.

In the beginning, some of the healthcare providers held the view that digital night surveillance was a threat to patient privacy and dignity. This view seemed to change, however, and the resistance that emerged to this perceived threat seemed to convert into a moral argument in favour of digital night surveillance. As the experience with the technology grew, a critical view on previous prac- tice emerged. The argument was that ordinary, regular night visits, including observation while the patient was asleep, might represent a far more serious invasion of priv- acy and violation of dignity than a digital signal on the nurses' phone when assistance was needed. Digital night surveillance made it possible not to disturb the person in question unnecessarily, for instance, avoid waking him or her up at night in order to perform intimate actions, like adult diaper checks.

Resistance due to issues of justice A final moral issue that was raised among healthcare providers that gave rise to some resistance to the project was the question of equal access to and just distribution of the technology. In this project the technology was not implemented on a large scale and accessible to all. Not all patients that could have benefitted from the technol- ogy had access to it, and some patients moved into nurs- ing facilities where the technology was installed, without using it. This was sometimes hard to explain to relatives, but did not interfere with the innovation and implemen- tation process. In general, there was a change during the project period

from scepticism and resistance, to a broader acceptance, and to some degree even enthusiasm, on moral grounds among healthcare personnel. One of the technology devel- opers also made this observation:

It has quite clearly been a change here, and the best example is that some years ago we were fighting against the perception that it was unethical to use technology here, that this was all about the warm hands (…) whereas now the norm is that it is unethical to not use the technology.

Discussion Four main forms of resistance – and perceived threats This exploration of resistance to an implementation of welfare technology in municipal healthcare services has displayed a series of resistance behaviours, mostly passive and uncooperative, among different groups of agents – management, IT management, support staff, technology developers and healthcare providers. Four main categories of resistance were identified: 1) organizational resistance, including management resistance to participatory pro- cesses and necessary competence building, 2) cultural resistance, including resistance to cooperation and co-

Nilsen et al. BMC Health Services Research (2016) 16:657 Page 9 of 14

creation across professional groups, 3) technological resistance, including resistance represented by the muni- cipal IT infrastructure itself, and 4) ethical resistance, in- cluding healthcare providers’ resistance to implementing the new technology. The resistance seemed to emerge from a variety of perceived threats, partly parallel to and partly across the four categories of resistance: a) threats to stability and predictability (fear of change), b) threats to role and group identity (fear of losing power or control), and c) threats to basic healthcare values (fear of losing moral or professional integrity).

Implementation ambivalence Summing up these findings, it might seem that there was a massive resistance to technology implementation. This was not the case. Except for the quite strong and persistent resistance represented by the IT infrastruc- ture, most of the identified forms of resistance were pas- sive more often than active, weak rather than strong, subtle rather than outspoken. Some of the initial scepti- cism and resistance even became the opposite, such as resistance due to moral concerns, which to some degree transformed into moral motivation and arguments for applying the new technology when the concerns were met and the technology worked safely. In addition, parallel to the variety of resistance, there were also considerable positive interest, energy and enthusiasm among the par- ticipants. In other words, the exploration of resistance to co-creation and/or implementation also unveiled that the variety of forms of resistance most often were intertwined with the opposite, a motivation to co-create and imple- ment the technology. To various degrees throughout the project period, such implementation ambivalence charac- terized most of the participants, both developers, IT personnel, healthcare providers, projects leaders and municipal managers.

Productive resistance It seems like both resistance and ambivalence were productive as sources of creativity and co-creation. For example, the resistance that emerged from the threat of technological instability, unpredictability and lack of safety also triggered healthcare providers’ and developers’ creativity and cooperation to improve the technology and service. The healthcare providers helped co-create the technology through resisting the use of a technology that was not fully developed. Likewise, the technology devel- opers helped co-create new service routines through resisting the acceptance of a non-technological practice. This may be characterized as ‘productive resistance’ [23]. In this project, productive resistance emerged from two elements: a technology or practice that failed and a co-creation process design that aimed to develop un- finished products or services [23]. The resistance became

a constructive force that pushed the innovation process forward. The main reasons why much of the identified re- sistance in this project seemed to turn productive were probably 1) the use of an orchestrator, external to both of the participating ‘camps’, and 2) a workshop design, func- tioning as a learning network where all parties could meet regularly, share experiences and reflect openly together [53, 54]. Orchestrating the workshops as processes of ‘translation’ between the different professional cultures [55] was key to developing trust, enhancing knowledge of each other’s perspectives and making resistance turn productive.

Organizational resistance The classical theoretical approach to resistance in organi- zations has a negative outlook on resistance, seeing resist- ance mainly as a counter-force to power and control mechanisms [24, 27]. The active resistance acted by the municipal IT support department as well as a more passive resistance from the management in the health- care institutions may have been motivated by the fear of losing power. This was intertwined with the “struggle” be- tween stability, safety and predictability on one hand, and co-creation on the other. Participation in a pilot project evoked a certain resistance in itself, since the technology was under development and in need of improvement. This was the exact purpose of the project, but included none- theless an element of dynamism and insecurity that was contrary to the services’ need for control and stability. The IT support departments, in particular, appeared to have a low degree of tolerance towards insecurity and loss of control.

Cultural resistance Cultural resistance refers to both the communication problems between healthcare providers and technology developers, as well as the resistance that emerged from the implementation of the project’s feature as a co-creation project [21]. Even though the innovators contributed to “promulgation and spread of novelties” [29:1], the commu- nication difficulties appeared to be based in both the lack of shared vocabulary and in a mutual prejudice of the other sphere (technological vs healthcare). These cultural ten- sions as well as a mutual foreignness to co-creation [20], evoked resistance to the role of co-creator in both ‘camps’. Cultural differences and lack of redundant knowledge are challenging barriers to overcome in the implementation of technology [56], and the orchestrator who designed a translation process in both directions proved to be justified [42, 55].

Technological resistance Concerning technological resistance, there were two surprising findings. The first was that the municipal IT

Nilsen et al. BMC Health Services Research (2016) 16:657 Page 10 of 14

infrastructure in itself represented a serious resistance to the implementation process. From our material, the IT infrastructure emerged as perhaps the most unco- operative entity of all, a subject of resistance in its own right. This might seem strange, considering that sub- jects of resistance normally are individuals, groups of persons or organizations. However, the observation that an artefact can serve as a social-relational function is not new. The Actor Network Theory provides a corrective to the usual social scientific focus upon human beings by “directing attention to the significance of nonhumans in social life” ([57]:109) – in this case the IT infrastructure, obstructing the process of co-creation and innovation. The second surprising finding was the passive resist-

ance represented by the fact that nothing was done on the management level of the municipalities to include the IT departments at an initial stage, in order to pre- pare the IT system and support staff for the co-creation and implementation process. This is even more surpris- ing considering the well-known fact that the municipal IT infrastructure would play a crucial role, and that implementation of welfare technologies is high on the political agenda. We have interpreted this omission as passive management resistance to participative processes. This finding is in line with research on collaborative innovation projects in the public sector, identifying co- initiation as a success factor, suggesting that public leaders and managers may be reluctant to co-initiation because of fear of losing power [58]. We can only speculate as to what, in this case, the perceived threat might have been – fear of losing power, financial consequences or something else. Whatever the reason might be, the finding suggests that more attention should be drawn to the importance of co-initiation and participative processes at an initial stage when planning complex municipal innovation and imple- mentation projects. The resistance from the IT support staff can be char-

acterized as active resistance and was at times perceived by other stakeholders (healthcare providers and technol- ogists) as aggressive [36]. For the managers, it appeared to be due to a poor understanding of their role in the implementation process [59]. The management did not take an active interest in the implementation, and their lack of interest can be categorized as a passive resistance that manifested in practice [33, 36].

Ethical resistance Ethical resistance refers to resistance emerging from re- flection on perceived threats to basic healthcare values and professional ethics [60, 61]. Four main perceived threats were identified: 1) threats to patient safety, 2) threats to the quality of care, 3) threats to patient priv- acy and dignity, and 4) threats to equal access and just

distribution. These findings are consistent with previous research with regard to the development and use of wel- fare technologies [3, 5, 62, 63]. Indirectly these moral concerns seem to represent arguments that may be found in healthcare (organizational and clinical) ethics. These are based on ethical theories, like the moral obli- gations to secure patients’ safety and rights (duty ethics), to consider moral implications, such as possible harm to patients' privacy, dignity, autonomy and integrity (consequentialism), and to protect one’s integrity as a morally mindful, caring and professional healthcare worker (virtue ethics) [64, 65]. Ethical resistance concerns the core of the healthcare

providers’ professional practice, including how she uses her knowledge, skills and senses when she sees, touches, smells and speaks to the patient. Changing circumstances in the form of increased use of technology is perceived to alter and discipline the professional work [66], and profes- sionals face new threats that have to be managed. These can be fear of not being a good healthcare provider or a caring institution and a threat to their identity as health- care providers. Due to the changing circumstances, the content of the professionalism is contested. The concept of ethical resistance might help leaders to

recognize that this kind of resistance represents cues to moral concerns that have to be identified and solved in order to prevent adverse events and to help transform staff resistance into motivation. The concept might also help leaders avoid the psychologization fallacy, to con- fuse the ethical resistance of putting values at risk with the psychological resistance of change as a negative force that has to be overcome. It might also help leaders develop their ethical leadership skills [67], by using ethical resistance as a golden opportunity of de- tecting and managing moral risk and improving the moral quality of both the implementation process and final result [67]. In concluding the discussion, according to the infor-

mants, the initial resistance and scepticism of the new technology was replaced to a certain degree by a posi- tive attitude towards implementation of the technology. We see three partly overlapping explanations for this. One might be adaptation, meaning that the healthcare providers got used to the technology and learned that it was helpful, not harmful [33, 68]. Another explanation might be ethical reflection upon the experience that the surveillance technology proved to enhance patient safety and reduce intrusions of privacy at night. A third explanation might be the facilitated interaction and knowledge sharing, including ethical reflection, during workshops and other meetings. This might have contrib- uted both to adaptation, solutions to moral problems and a feeling of connectedness, competence and coping, factors associated with motivation [69, 70].

Nilsen et al. BMC Health Services Research (2016) 16:657 Page 11 of 14

Implications for practice In planning the implementation of welfare technology in municipal organizations one should consider a) the IT infrastructure, b) co-initiation, c) translation spaces and d) use of an external orchestrator. Managers should consider ethical resistance as product-

ive, and promote co-creation between care personnel and technologists in order to meet the moral concerns.

Issues for further research In studies as the one at hand, many factors influence the context. In order to reduce complexity, we have omitted several factors. Central and important stakeholders like the patients and next of kin have not been included in the study. This is because we wanted to focus on the employees, but at the same time, we recognize the pa- tient and his/her family as the real end user of the wel- fare technology. Focus on the patient and families will need to be included in future studies.

Conclusion This study identifies forms of resistance that appear to slow down the implementation of technology in a healthcare setting, especially resistance to participate in collaborative processes, resistance connected to the IT infrastructure and resistance arising from ethical con- cerns. It contributes to the body of literature on resist- ance to technology in a municipal healthcare setting, since the majority of extant research on resistance in healthcare has been performed in hospitals. Further- more, the technology in question is sensor technology in combination with a web-based portal, which is also atypical for studies within the field. Contrary to what might be expected from previous

findings (e.g. [8]), we found that resistance to surveil- lance technology on a general note was not significant, and the healthcare providers perceived the new tech- nology as a threat only to a low extent. In the long term, this could be explained by involvement in the co- creation process and motivated by a perception that a positive attitude towards this technology is appropriate and “modern”, rather than seeing technology in itself as a threat. The healthcare providers also appear to conceive the advantages and the future use of welfare technology. Theoretically, the study contributes by identifying

resistance categories, coining the concept of ethical resistance and focusing on productive resistance. Re- sistance appears to play a productive role when the implementation is organized as a co-creation process. The study has shown that resistance changes character over time and that it is not solely a negative phenomenon, as it contributes to development and innovation through the friction it creates.

Additional file

Additional file 1: Interview guide for semi structured focus and individual interviews. (DOCX 14 kb)

Abbreviations EP: Entered the project; FG: Focus group interviews; HIT: Health information technologies; II: Individual interviews; IS: Information systems; IT: Information technology; PO: Participatory observation; SME: Small and medium-sized enterprises; WF: Welfare technology; WS: Workshops

Acknowledgement We would like to thank the participants who willingly took part in our study.

Funding The study was funded by the Regional Research Fund in Norway (project numbers 229883 and 234978). The funding body did not have any role in the study design, the collection, analysis, and interpretation of the data, in the writing of the paper, nor in the decision to submit the paper for publication.

Availability of data and materials An interview guide is available (Additional file 1). The data collected for this study consists of transcribed interviews and field notes. These qualitative data will not be made available for privacy reasons.

Authors’ contributions All authors made significant contributions to the manuscript. The study was conceived by ERN, and was drafted in close cooperation with JD, HE and MKG. ERN, JD, HE, MKG and TE collected data and contributed to the analysis. The manuscript was written by ERN, JD, HE, MKG and TE. All authors read and approved of the final manuscript.

Competing interests The authors declare that they have no competing interests.

Consent for publication Not applicable.

Ethics approval and consent to participate The project has been approved by the NSD, the Norwegian Data Service for Social Sciences (ethical approval no. 34831). The participants signed an informed consent form. The data are anonymized in the presentations.

Author details 1The Science Centre Health and Technology, School of Business, University College of Southeast Norway, Postboks 235, N-3603 Kongsberg, Norway. 2The Science Centre Health and Technology, Faculty of Health Sciences, University College of Southeast Norway, Postboks 235, N-3603 Kongsberg, Norway.

Received: 12 March 2016 Accepted: 9 November 2016

References 1. Commission of the European Communities: Together for health: a strategic

approach for the EU 2008–2013. Brussels: European Commission Brussels; 2007. 2. World Health Organization: World report on ageing and health. 2015.

Luxembourg: World Health Organization; 2015. p. 1–260. 3. Niemeijer AR, Frederiks BJ, Riphagen II, Legemaate J, Eefsting JA, Hertogh

CM. Ethical and practical concerns of surveillance technologies in residential care for people with dementia or intellectual disabilities: an overview of the literature. Int Psychogeriatr. 2010;22(07):1129–42.

4. Bharucha AJ, London AJ, Barnard D, Wactlar H, Dew MA, Reynolds CF. Ethical considerations in the conduct of electronic surveillance research. J Law Med Ethics. 2006;34(3):611–19.

5. Hofmann B. Ethical challenges with welfare technology: a review of the literature. Sci Eng Ethics. 2013;19(2):389–406.

6. Berwick DM. Disseminating innovations in health care. JAMA. 2003;289(15): 1969–75.

Nilsen et al. BMC Health Services Research (2016) 16:657 Page 12 of 14

7. Waring J, Currie G. Managing Expert Knowledge: Organizational challenges and managerial futures for the UK medical profession. Organ Stud. 2009;30(7):755–78.

8. Bao Y. Organizational resistance to performance-enhancing technological innovations: a motivation-threat-ability framework. J Bus Ind Mark. 2009;24(2):119–30.

9. West MA, Sacramento CA. Flourishing in Teams. Developing Creativity and Innovation. In: Henry J, editor. Creative Management and Development (2006). London: Sage; 2006. p. 25–44.

10. West MA, Altink WM. Innovation at work: Individual, group, organizational, and socio-historical perspectives. Eur J Work Organ Psychol. 1996;5(1):3–11.

11. Anderson N, De Dreu CK, Nijstad BA. The routinization of innovation research: A constructively critical review of the state‐of‐the‐science. J Organ Behav. 2004;25(2):147–73.

12. Länsisalmi H, Kivimäki M, Aalto P, Ruoranen R. Innovation in healthcare: a systematic review of recent research. Nurs Sci Q. 2006;19(1):66–72.

13. Fleuren MAH, Paulussen TGWM, Dommelen P, Van Buuren S. Towards a measurement instrument for determinants of innovations. Int J Qual Health Care. 2014;20:1–10.

14. Rogers EM. Diffusion of innovations. New York: Free Press; 2003. 15. Beer M, Nohria N. Cracking the code of change. Harv Bus Rev. 2000;78(3):133–41. 16. Kapsali M. How to implement innovation policies through projects

successfully. Technovation. 2011;31(12):615–26. 17. Angst C, Agarwal R. Adoption of electronic health records in the presence

of privacy concerns: the elaboration likelihood model and individual persuasion. MIS Q. 2009;33(2):339–70.

18. Bates DW. Physicians and ambulatory electronic health records. Health Aff. 2005;24(5):1180–89.

19. Simon SR, Kaushal R, Cleary PD, Jenter CA, Volk LA, Poon EG. Correlates of electronic health record adoption in the office practices: A statewide survey. J Am Med Inform Assoc. 2007;14(1):110–17.

20. Neale MR, Corkindale DR. Co-developing products: Involving customers earlier and more deeply. Long Range Plan. 1998;31(3):418–25.

21. Prahalad CK, Ramaswamy V. Co-creation experiences: The next practice in value creation. J Interact Mark. 2004;18(3):5–14.

22. Knorr-Cetina K. Epistemic cultures: how the sciences make knowledge. Cambridge: Harvard University Press; 1999.

23. Courpasson D, Dany F, Clegg S. Resisters at work: Generating productive resistance in the workplace. Organ Sci. 2012;23(3):801–19.

24. Mumby DK. Theorizing resistance in organization studies A dialectical approach. Manag Commun Q. 2005;19(1):19–44.

25. Hollander JA, Einwohner RL. Conceptualizing resistance. Sociol Forum. 2004;19(4):533–54.

26. Samhan B, Joshi KD. Resistance of Healthcare Information Technologies: Literature Review, Analysis, and Gaps. In: 2015 48th Hawaii International Conference on System Sciences. Washington: IEEE Computer Society; 2015;2992-3001.

27. Piderit SK. Rethinking resistance and recognizing ambivalence: a multidimensional view of attitudes toward organizational change. Acad Manag Rev. 2000;25(4):783–94.

28. Bintoro BPK, Simatupang TM, Putro US, Hermawan P. Actors’ interaction in the ERP implementation literature. Bus Process Manag J. 2015;21(2):222–49.

29. Barnett J, Vasileiou K, Djemil F, Brooks L, Young T. Understanding innovators’ experiences of barriers and facilitators in implementation and diffusion of healthcare service innovations: a qualitative study. BMC Health Serv Res. 2011;11(1):342.

30. Ho K, Bloch R, Gondocz T, Laprise R, Perrier L, Ryan D, Thivierge R, Wenghofer E. Technology-enabled knowledge translation: frameworks to promote research and practice. J Contin Educ Health Prof. 2004;24(2):90–9.

31. Lluch M. Healthcare professionals’ organisational barriers to health information technologies—A literature review. Int J Med Inform. 2011;80(12):849–62.

32. Wilson VJ, McCormack BG, Ives G. Understanding the workplace culture of a special care nursery. J Adv Nurs. 2005;50(1):27–38.

33. Lapointe L, Rivard S. A multilevel model of resistance to information technology implementation. MIS Q. 2005;29(3):461–91.

34. Iden J, Eikebrokk TR. Implementing IT Service Management: A systematic literature review. Int J Inf Manag. 2013;33(3):512–23.

35. Olsen NV. Incremental innovation: a way to handle friction? Int J Innov Regional Dev. 2011;3(2):174–85.

36. Coetsee L. From resistance to commitment. Public Administration Quarterly. 1999;23(2):204–22.

37. Bhattacherjee A, Hikmet N. Physicians’ resistance toward healthcare information technology: a theoretical model and empirical test. Eur J Inf Syst. 2007;16(6):725–37.

38. Øvretveit J, Scott T, Rundall TG, Shortell SM, Brommels M. Improving quality through effective implementation of information technology in healthcare. Int J Qual Health Care. 2007;19(5):259–66.

39. Denis J-L, Hébert Y, Langley A, Lozeau D, Trottier L-H. Explaining diffusion patterns for complex health care innovations. Health Care Manag Rev. 2002;27(3):60–73.

40. Lusch RF, Vargo SL. Service-dominant logic: reactions, reflections and refinements. Mark Theory. 2006;6(3):281–88.

41. Ranga M, Etzkowitz H. Triple Helix systems: an analytical framework for innovation policy and practice in the Knowledge Society. Ind High Educ. 2013;27(4):237–62.

42. Gausdal AH, Nilsen ER. Orchestrating innovative SME networks. The case of “Health Innovation”. J Knowl Econ. 2011;2(4):586–600.

43. Barlow J, Singh D, Bayer S, Curry R. A systematic review of the benefits of home telecare for frail elderly people and those with long-term conditions. J Telemed Telecare. 2007;3:72–179.

44. Lai CKY, Arthur DG. Wandering behaviour in people with dementia. J Adv Nurs. 2003;44:173–82.

45. Yin RK. Case study research. Design and Methods. 5th edition. Thousand Oaks: Sage Publications; 2014.

46. Winter R, Munn-Giddings C, editors. A handbook for action research in health and social care. London: Routledge; 2001.

47. Alvesson M, Sköldberg K. Reflexive methodology: New vistas for qualitative research. 2nd ed. SAGE: London, Thousand Oaks; 2009.

48. Alvesson M. Beyond Neopositivists, Romantics, and Localists. A reflexive approach to interviews in organizational research. Acad Manag Rev. 2003;28(1):13–33.

49. Kvale S, Brinkmann S. Interviews: learning the craft of qualitative research interviewing. Los Angeles: Sage; 2009.

50. Webb C, Kevern J. Focus groups as a research method: a critique of some aspects of their use in nursing research. J Adv Nurs. 2001;33(6):798–805.

51. Carter N, Bryant-Lukosius D, Dicenso A, Blythe J, Neville AJ. The use of triangulation in qualitative research. Oncol Nurs Forum. 2014;41(5):545–7.

52. Pope C, Ziebland S, Mays N. Qualitative research in health care: Analysing qualitative data. Br Med J. 2000;320(7227):114–6.

53. Nonaka I, Konno N. The concept of `Ba': Building a foundation for knowledge creation. Calif Manag Rev. 1998;40(3):40–54.

54. Wenger E, McDermott R, Snyder WM. Cultivating communities of practice, A guide to managing knowledge. Boston: Harvard Business School Press; 2002.

55. Wæraas A, Nielsen JA. Translation theory ‘Translated’: Three perspectives on translation in organizational research. Int J Manag Rev. 2016;18(3):236–70.

56. Nonaka I. A dynamic theory of organizational knowledge creation. Organ Sci. 1994;5(1):14–37.

57. Nimmo R. Actor-network theory and methodology: Social research in a more-than-human world. Methodological Innovat Online. 2011;6(3):108–19.

58. Sørensen E, Torfing J. Co-initiation of Collaborative Innovation in Urban Spaces. Urban Affairs Review. 2016; doi:10.1177/1078087416651936

59. Pollack J. The changing paradigms of project management. Int J Proj Manag. 2007;25(3):266–74.

60. ENDA. The European nurse directors’ proto-code of ethics and conduct. European Nurse Directors Association. 2011. http://www.enda-europe.com/ en/publications.html. Accessed 1 March 2016.

61. The ICN Code of Ethics for Nurses. International Council of Nurses. http://www.icn.ch/images/stories/documents/about/icncode_english. pdf. 2012. Accessed 1 March 2016.

62. Demiris G, Doorenbos AZ, Towle C. Ethical considerations regarding the use of technology for older adults: The case of telehealth. Res Gerontol Nurs. 2009;2(2):128–36.

63. Zwijsen SA, Depla MF, Niemeijer AR, Francke AL, Hertogh CM. Surveillance technology: An alternative to physical restraints? A qualitative study among professionals working in nursing homes for people with dementia. Int J Nurs Stud. 2012;49(2):212–19.

64. Eide T, Aadland E. Etikkhåndboka: for kommunenes helse- og omsorgstjenester 2. utgave [The ethics handbook for the municipalities’ health and care services. 2nd ed.]. Oslo: Kommuneforlaget; 2012.

65. Slowther A, Johnston C, Goodall J, Hope T. A practical guide for clinical ethics support. Oxford: The Ethox Centre, University of Oxford; 2004.

66. Noordegraaf M. Risky business: How professionals and professional fields (must) deal with organizational issues. Organ Stud. 2011;32(10):1349–71.

Nilsen et al. BMC Health Services Research (2016) 16:657 Page 13 of 14

67. Eide T, van Dulmen S, Eide H. Educating for ethical leadership through web- based coaching. A feasibility study. Nursing Ethics. May 24 2015. pii: 0969733015584399. Epub ahead of print.

68. Greenhalgh T, Stones R. Theorising big IT programmes in healthcare: Strong structuration theory meets actor-network theory. Soc Sci Med. 2010;70(9):1285–94.

69. Baard PP, Deci EL, Ryan RM. Intrinsic need satisfaction. A motivational basis of performance and Weil‐being in two work settings. J Appl Soc Psychol. 2004;34(10):2045–68.

70. Deci EL, Ryan RM. Self-determination theory: A macrotheory of human motivation, development, and health. Can Psychol. 2008;49(3):182.

• We accept pre-submission inquiries • Our selector tool helps you to find the most relevant journal • We provide round the clock customer support • Convenient online submission • Thorough peer review • Inclusion in PubMed and all major indexing services • Maximum visibility for your research

Submit your manuscript at www.biomedcentral.com/submit

Submit your next manuscript to BioMed Central and we will help you at every step:

Nilsen et al. BMC Health Services Research (2016) 16:657 Page 14 of 14

BioMed Central publishes under the Creative Commons Attribution License (CCAL). Under the CCAL, authors retain copyright to the article but users are allowed to download, reprint, distribute and /or copy articles in BioMed Central journals, as long as the original work is properly cited.

RESEARCH ARTICLE

Applications of artificial neural networks in

health care organizational decision-making: A

scoping review

Nida ShahidID 1,2*, Tim Rappon1, Whitney Berta1

1 Institute of Health Policy, Management and Evaluation, University of Toronto, Toronto, Canada, 2 Toronto

Health Economics and Technology Assessment (THETA) Collaborative, University Health Network, Toronto,

Canada

* [email protected]

Abstract

Health care organizations are leveraging machine-learning techniques, such as artificial

neural networks (ANN), to improve delivery of care at a reduced cost. Applications of ANN

to diagnosis are well-known; however, ANN are increasingly used to inform health care

management decisions. We provide a seminal review of the applications of ANN to health

care organizational decision-making. We screened 3,397 articles from six databases with

coverage of Health Administration, Computer Science and Business Administration. We

extracted study characteristics, aim, methodology and context (including level of analysis)

from 80 articles meeting inclusion criteria. Articles were published from 1997–2018 and orig-

inated from 24 countries, with a plurality of papers (26 articles) published by authors from

the United States. Types of ANN used included ANN (36 articles), feed-forward networks

(25 articles), or hybrid models (23 articles); reported accuracy varied from 50% to 100%.

The majority of ANN informed decision-making at the micro level (61 articles), between

patients and health care providers. Fewer ANN were deployed for intra-organizational

(meso- level, 29 articles) and system, policy or inter-organizational (macro- level, 10 arti-

cles) decision-making. Our review identifies key characteristics and drivers for market

uptake of ANN for health care organizational decision-making to guide further adoption of

this technique.

Introduction

As health care systems in developed countries transform towards a value based, patient-cen-

tered model of care delivery, we face new complexities relating to improving the structure and

management of health care delivery; for example, improving integration of processes in care

delivery for patient-centered chronic disease management [1]. Artificial intelligence lies at the

nexus of new technologies with the potential to deliver health care that is cost-effective and

appropriate care in real-time, manage effective and efficient communication among multidis-

ciplinary stakeholders, and address non-traditional care settings, the evolving heathcare work-

place and workforce, and the advent of new and disparate health information systems. With

PLOS ONE | https://doi.org/10.1371/journal.pone.0212356 February 19, 2019 1 / 22

a1111111111

a1111111111

a1111111111

a1111111111

a1111111111

OPEN ACCESS

Citation: Shahid N, Rappon T, Berta W (2019)

Applications of artificial neural networks in health

care organizational decision-making: A scoping

review. PLoS ONE 14(2): e0212356. https://doi.

org/10.1371/journal.pone.0212356

Editor: Olalekan Uthman, The University of

Warwick, UNITED KINGDOM

Received: October 4, 2018

Accepted: January 31, 2019

Published: February 19, 2019

Copyright: © 2019 Shahid et al. This is an open access article distributed under the terms of the

Creative Commons Attribution License, which

permits unrestricted use, distribution, and

reproduction in any medium, provided the original

author and source are credited.

Data Availability Statement: All relevant data are

within the manuscript and its Supporting

Information files.

Funding: The authors received no specific funding

for this work.

Competing interests: The authors have declared

that no competing interests exist.

the rapid uptake of artificial intelligence to make increasingly complex decisions across differ-

ent industries, there are a multitude of solutions capable of addressing these health care man-

agement challenges; however, there is a paucity of guidance on selecting appropriate methods

tailored to the health care industry[2].

Global health care expenditure is expected to reach $8.7 trillion by 2020, driven by aging

populations growing in size and disease complexity, advancements made in medical treat-

ments, rising labour costs and the market expansion of the health care industry. Many health

systems are reported to struggle with updating aging infrastructure and legacy technologies

with already limited capital resources. In an effort toward moving to value-based care, deci-

sion-makers are reported to be strategically shifting the focus to understanding and better

alignment of financial incentives for health care providers in order to bear financial risk; popu-

lation health management including analyses of trends in health, quality and cost; and adop-

tion of innovative delivery models for improved processes and coordination of care.

Health care organizations are required to be increasingly strategic in their management due

to a variety of system interdependences such as emerging environmental demands and com-

peting priorities, that can complicate decision-making process [3]. According to economy the-

ory, most organizations are risk-aversive [4] and decision-makers in health care can face issues

related to culture, technology and risk when making high-risk decisions without the certainty

of high-return [4, 5]. Patient care and operations management requires the interaction of mul-

tiple stakeholders, for example clinicians, front-line/middle managers, senior level executives

to make decisions on a clinical (e.g. diagnosis, treatment and therapy, medication prescription

and administration), and non-clinical (e.g. budget, resource allocation, technology acquisition,

service additions/reductions, strategic planning) [6].

A white paper published by IBM suggests that with increasing capture and digitization of

health care data (e.g. electronic medical records and DNA sequences), health care organizations

are taking advantage of analyzing large sets of routinely collected digital information in order to

improve service and reduce costs [7]. Reported examples include analyzing clinical, financial and

operational data to answer questions related to effectiveness of programs, making predictions

regarding at-risk patients. The global market for health care predictive analytics is projected was

valued at USD 1.48 billion in 2015 and expected to grow at a rate of 29.3% (compound annual

growth rate) by 2025 [8]. Similarly, global revenue of $811 million is expected to increase 40%

(Compound Annual Growth Rate) by 2021 due the artificial intelligence (AI) market for health

care applications. A subfield of AI, machine learning-as-a-service-market (MLaaS), is expected to

reach $5.4 billion by 2022, with the health care sector as a notable key driver [9].

A recent survey of AI applications in health care reported uses in major disease areas such

as cancer or cardiology and artificial neural networks (ANN) as a common machine learning

technique [10]. Applications of ANN in health care include clinical diagnosis, prediction of

cancer, speech recognition, prediction of length of stay [11], image analysis and interpretation

[12] (e.g. automated electrocardiographic (ECG) interpretation used to diagnose myocardial

infarction [13]), and drug development[12]. Non-clinical applications have included improve-

ment of health care organizational management [14], prediction of key indicators such as cost

or facility utilization [15]. ANN has been used as part of decision support models to provide

health care providers and the health care system with cost-effective solutions to time and

resource management [16].

Rationale

Despite its many applications and, more recently, its prominence [17], there is a lack of coher-

ence regarding ANN’s applications and potential to inform decision making at different levels

Applications of ANN in health care organizational decision-making: A scoping review

PLOS ONE | https://doi.org/10.1371/journal.pone.0212356 February 19, 2019 2 / 22

in health care organizations. This review is motivated by a need for a broad understanding the

various applications of ANN in health care and aids researchers interested in bridging the dis-

ciplines of organizational behaviour and computer science. Considering the sheer abundance

in reported use and complexity of the area, it can be challenging to remain abreast of the new

advancements and trends in applications of ANN [18]. Adopters of ANN or researchers new

to the field of AI may find the scope and esoteric terminology of neural computing particularly

challenging [18]. Literature suggests that current reviews on applications of ANN are limited

in scope and generally focus on a specific disease [19] or a particular type of neural network

[20], or they are too broad (i.e. data mining or AI techniques that can include ANN but do not

offer insights specific to ANN) [10]. The overarching goal of this scoping review is to provide a

much-needed comprehensive review of the various applications of ANN in health care organi-

zational decision-making at the micro-, meso-, and macro-levels. The levels pertain to deci-

sions made on the (micro) level of individual patients, or on a (meso) group level (e.g.

departmental or organizational level) where patient preference may be important but not

essential; and on a wider (macro) level by large groups or public organizations related to allo-

cation or utilization of resources where decisions are based on public interest and reflective of

society as a whole [21]. By means of this review, we will identify the nature and extent of rele-

vant literature and describe methodologies and context used.

Overview

According to an overview by Kononenko (2001), as a sub-field of AI, machine learning pro-

vides indispensable tools for intelligent data analysis. Three major branches of machine learn-

ing have emerged since electronic computers came in to use during the 1950s and 1960s:

statistical methods, symbolic learning and neural networks [22]. ANN have been successfully

used to solve highly complex problems within the physical sciences and as of late by scholars

in organizational research as digital tools enabling faster processes of data collection and pro-

cessing [23]. As practical and flexible modelling tools, ANN have an ability to generalize pat-

tern information to new data, tolerate noisy inputs, and produce reliable and reasonable

estimates [23]. ANN belong to a wide class of flexible nonlinear regression and discriminant

models, data reduction models, and nonlinear dynamical systems [24]. ANN are similar to sta-

tistical techniques including generalized linear models, nonparametric regression and discrim-

inant analysis, or cluster analysis [24]. As a statistical model, it’s general composition is one

made of simple, interconnected processing elements that are configured through iterative

exposure to sample data [23]. Its application is particularly valuable under one or more of sev-

eral conditions: when sample data show complex interaction effects or do not meet parametric

assumptions, when the relationship between independent and dependent variables is not

strong, when there is a large unexplained variance in information, or in situations where the

theoretical basis of prediction is poorly understood [23]. ANN architectures are commonly

classified as feed-forward neural networks (e.g. single-layer perceptron, multi-layer percep-

tron, radial basis function networks) or feed-back, or otherwise referred to as recurrent neural

networks (e.g. Competitive networks, Kohonen’s self-organizing maps, Hopfield networks)

[25].

Artificial neural networks

Originally developed as mathematical theories of the information-processing activity of bio-

logical nerve cells, the structural elements used to describe an ANN are conceptually analogous

to those used in neuroscience, despite it belonging to a class of statistical procedures [23].

Applications of ANN in health care organizational decision-making: A scoping review

PLOS ONE | https://doi.org/10.1371/journal.pone.0212356 February 19, 2019 3 / 22

Basics

ANN can have single or multiple layers [23], and consist of processing units (nodes or neu-

rons) that are interconnected by a set of adjustable weights that allows signals to travel through

the network in parallel and consecutively[13, 26]. Generally ANN can be divided in to three

layers of neurons: input (receives information), hidden (responsible for extracting patterns,

perform most of internal processing), and output (produces and presents final network out-

puts) [27].

A review by Agatonovic-Kustrin & Beresford (2000) describes neural computation to be

powered from the connection of its neurons and that each neuron has a weighted input, trans-

fer function and a single output. The authors state that the neuron is activated by the weighed

sum of inputs it receives and the activation signal passes through a transfer function to pro-

duce a single output. The transfer functions, the learning rule and the architecture determine

the overall behaviour of the neural network [26].

Architecture

Sharma & Chopra (2013) describe the two most common types of neural networks applied in

management sciences to be the feed-forward and recurrent neural networks (Fig 1) in compar-

ison with feed-forward networks common to medical applications [28, 29]. A feed-forward

network can be single-layered (e.g. Perceptron, ADALINE) or multi-layered (e.g. Multilayer

Perceptron, Radial Basis Function) [27, 30]. Sharma & Chopra (2013) describe information

flow in feed-forward networks to be unidirectional from input layer, through hidden layers to

the output layer, without any feedback. Whereas, a recurrent or feedback network involves

Fig 1. Conceptual model of a feed-forward and recurrent neural network.

https://doi.org/10.1371/journal.pone.0212356.g001

Applications of ANN in health care organizational decision-making: A scoping review

PLOS ONE | https://doi.org/10.1371/journal.pone.0212356 February 19, 2019 4 / 22

dynamic information processing having at least one feedback loop, using outputs as feedback

inputs (e.g. Hopfield) [27, 30]. Fig 1 illustrates the two types of networks with three layers

(input, hidden and output).

Learning

In an overview of basic concepts, Agatonovic-Kustrin & Beresford (2000) describe ANN

gather knowledge by detecting patterns and relationships in data and “learn” through experi-

ence. The authors state an artificial neural network learns by optimizing its inner unit connec-

tions in order to minimize errors in the predictions that it makes and to reach a desired level

of accuracy. New information can be inputted into the model once the model has been trained

and tested [26]. Also referred to as the generalized delta rule, backpropagation refers to how an

ANN is trained or ‘learns’ based on data. It uses an iterative process involving six steps: (i) sin-

gle case data is passed to input later, output is passed to the hidden layer and multiplied by the

first set of connection weights; (ii) incoming signals are summed, transformed to output and

passed to second connection weight matrix; (iii) incoming signals are summed, transformed

and network output is produced; (iv) output value is subtracted from known value for that

case, error term is passed backward through network; (v) connection weights are adjusted in

proportion to their error contribution; (vi) modified connection weights saved for next cycle,

next case input set queued for next cycle [23]. Sharma & Chopra (2013) broadly classify train-

ing or ‘learning’ methods in ANN into three types: supervised, unsupervised and reinforced

learning. In supervised learning, every input pattern used to train the network is associated

with an output pattern. The error in computed and desired outputs can be used to improve

model performance. In unsupervised learning, the network learns without knowledge of

desired output and by discovering and adapting to features of the input patterns. In reinforce-

ment learning, the network is provided with feedback on if computation performance without

presenting the desired output [30].

Artificial neural networks and regression models

Neural networks are similar to linear regression models in their nature and use. They are com-

prised of input (independent or predictor variable) and output (dependent or outcome vari-

able) nodes, use connection weights (regression coefficients), bias weight (intercept

parameters) and cross-entropy (maximum likelihood estimation) to learn or train (parameter

estimation) a model [31]. ANN learn to perform tasks by using inductive learning algorithms

requiring massive data sets [18]. A working paper on the use of ANN in decision support sys-

tems states that the structure, quality and quantity of data used is critical for the learning pro-

cess and that the chosen attributes must be complete, relevant, measurable and independent

[18]. The authors further observe that in business applications, external data sources (e.g.

industry and trade databases) are typically used to supplement internal data sources.

Classification and prediction modelling

In the book entitled ‘Data Mining: Concepts and Techniques’, classification is defined as the

process of finding a model that describes and distinguishes data classes or concepts based on

analysis of a set of training data [32]. The authors write that models called classifiers predict

categorical class labels and can be used to predict the class label of objects for which the class label is unknown. Furthermore, the process is described to consist of a learning step (when a

classification model is constructed) and a classification step (when a model is used to predict

class labels for a given data). Methods include naïve Bayesian classification, support vector machines, and k-nearest-neighbour classification [32]. Han et al. (2012) suggest that

Applications of ANN in health care organizational decision-making: A scoping review

PLOS ONE | https://doi.org/10.1371/journal.pone.0212356 February 19, 2019 5 / 22

applications can broadly include fraud detection, target marketing, performance prediction,

manufacturing and medical diagnosis.

The available data is divided into two sets for cross-validation: a training set used to develop

a model and a test set, used to evaluate the model’s performance [33, 34]. Appropriate data

splitting is a technique commonly used in machine learning in order to minimize poor gener-

alization (also referred to as over-training or over-fitting) of models [34]. Using more training

data improves the classification model, whereas using more test data contributes to estimating

error accurately [35]. Although a 70:30 ratio can typically be used for training/testing size [36],

various statistical sampling techniques ranging from simple (e.g. simple random sampling,

trial-and-error) to more deterministic (e.g. CADEX, DUPLEX) can be used to split the data

depending on the goals and complexity of the problem [34].

Han and colleagues (2012) write that where classification predicts categorical labels, regres-

sion is used to predict missing or unavailable numerical data values (rather than discrete class labels). The authors describe regression analysis as a statistical methodology often used for

numeric prediction and encompasses identification of distribution trends based on available

data. An example of numeric prediction is when a model is constructed to predict a continu-

ous-valued function or ordered value (as opposed to a class label). Such a model is called a pre-

dictor model and typically uses regression analysis [32].

ANN can be used to perform nonlinear statistical modeling and provide new alternatives to

logistic regression, the most commonly used method for developing predictive models for

dichotomous outcomes in medicine [31]. Users require less formal statistical training and the

networks are able to detect complex non-linear relationships and interactions between depen-

dent and independent variables. ANN can combine and incorporate literature-based and exper-

imental data to solve problems [26]. Other advantages of ANN, relative to traditional predictive

modeling techniques, include fast and simple operation due to compact representation of

knowledge (e.g., weight and threshold value matrices), the ability to operate with noisy or miss-

ing information and generalize to similar unseen data, the ability to learn inductively from

training data and process non-linear functionality critical to dealing with real-word data [37].

Although ANN do not require knowledge of data source, they require large training sets due

to the numerous estimated weights involved in computation [26]. They may require lengthy

training times and the use of random weight initializations may lead to different solutions [37].

Despite successful applications, ANN remain problematic in that they offer us little or no insight

into the process(es) by which they learn or the totality of the knowledge embedded in them [38].

Several limitations of ANN are identified in the literature: they are limited in their ability to explic-

itly identify possible causal relationships, they are challenging to use in the field, they are prone to

over fitting, model development is empirical potentially requiring several attempts to develop an

acceptable model [37], and there are methodological issues related to model development [31]. In

comparing advantages and disadvantages of using ANN to predict medical outcomes, Tu (1996)

suggests that logistic regression models can be disseminated to a wider audience, whereas ANN

models are less transparent and therefore can be more difficult to communicate and use. Even if

published and made available, the connection weight matrices used in ANN for training a data set

may be large and difficult to interpret for others to make use of, whereas logistic regression coeffi-

cients can be published for any end user to be able to calculate [31].

Methods

The Arksey & O’Malley framework (2005) was adopted to identify the (i) research question,

(ii) relevant studies, (iii) select studies, (iv) chart the data and (v), collate, summarize and pres-

ent findings.

Applications of ANN in health care organizational decision-making: A scoping review

PLOS ONE | https://doi.org/10.1371/journal.pone.0212356 February 19, 2019 6 / 22

Search strategy

Due to the cross-disciplinary nature of our query, the search strategy was designed to identify

literature from multiple databases according to the key disciplines of Health Administration

(Medline and Embase), Computer Science (ACM Digital Library and Advanced Technologies

& Aerospace Database), and Business and Management (ABI/Inform Global and JSTOR). The

selection of the three disciplines reflects the core concepts embedded in our research question:

‘what are the different applications of ANN (Computer Science) in health care organizational

decision-making (Health Administration and Business Management)?’

In consultation with a librarian, a comprehensive search syntax was built on the concepts of

‘artificial neural networks’ applied in ‘health care organizational decision-making’ and tailored

for each database for optimum results. The final search syntax was based on search terms

refined through an iterative process involving examination of a preliminary set of results to

ensure relevance (S1 Appendix). The search strategy was limited to peer-reviewed publications

in English without limitation to the year of publication up until the time of our search (January

2018). Our background search did not identify seminal paper(s) published or advancements

related to our research question, thereby justifying the rationale for not limiting the search to a

specicic start date.

Data collection

Screening of articles occurred in two stages. Identified articles were de-duplicated and

imported to EndNote as a reference manager and to Covidence, a web-based platform, for

screening. The screening inclusion and exclusion criteria were built iteratively via consensus

(NS, TR and WB) (Table 1). Titles and abstracts were first screened to include articles with

keywords related to and/or in explicit reference to artificial neural networks. Articles were

excluded if there was no explicit reference to artificial neural networks; the application was not

in the health care domain or context of health care organizational decision-making, or was not

a publication that was peer-reviewed (e.g. grey literature e.g. conference abstracts and papers,

book reviews, newspaper or magazine articles, teaching courses). Table 1 lists the criteria used

to screen, include or exclude articles in the review.

Subsequently, a full-text review of articles that met the initial screening criteria was con-

ducted on basis of relevance and availability of information for data extraction. In addition to

independent review and extraction of articles, two coders (NS and TR) extracted data from a

subset of articles for consensus, minimization of error, and clarity between reviewers regarding

Table 1. Screening inclusion, exclusion criteria.

Inclusion criteria Exclusion criteria

Titles and

abstracts

Explicit reference to keywords: neural network;

artificial neural network; ANNs;

Does not make explicit reference to artificial

neural networks within the context of healthcare

or medicineMust make reference to ANN if any type of

artificial intelligence or machine learning

techniques used, (e.g. Fuzzy logic, Bayesian

statistics and Self-Organizing Maps, back-

propagation; prediction model; unsupervised

learning)

Publication

Type

Peer-reviewed empirical or theoretical work

(e.g. Journal articles, reports)

Not based on empirical or theoretical work (e.g.

book reviews, newspaper article, course

material); conference papers and abstracts

Setting or

Context

Application in domain of Healthcare and/or

Medicine

Application was not directly related to healthcare

organizational decision making (e.g. speech

recognition)

https://doi.org/10.1371/journal.pone.0212356.t001

Applications of ANN in health care organizational decision-making: A scoping review

PLOS ONE | https://doi.org/10.1371/journal.pone.0212356 February 19, 2019 7 / 22

the choice of data selected for extraction. Information related to study characteristics, aim,

methodology (application, taxonomy, accuracy) and context including organizational level of

analysis (micro-, meso- and macro-) was collected and entered into Microsoft Excel for cate-

gorization and descriptive analysis. Applications of ANN to make decisions directly between

providers and patients was categorized as ‘micro’, any decisions made by a larger group and

not directly related to a patient was categorized as ‘meso’, and decisions beyond an organiza-

tional group (i.e. across different institutions, a system or countries) was categorized as

‘macro’ level of decision-making.

Results

Overall, 3,457 articles were imported for screening, out of which (after removal of duplicates)

3,397 were screened for titles and abstracts to give a total of 306 articles used for full-text

review (Fig 2). Articles were excluded from data collection for reasons such as: there being no

Fig 2. Review process overview. �Articles excluded for the following reasons: Not ANN or suitable synonym (n = 93), use of ANN

unrelated to healthcare organizational decision-making (n = 70), based on iterated exclusion criteria (n = 45), not based on

empirical or theoretical research (n = 9), could not access full-text (n = 9).

https://doi.org/10.1371/journal.pone.0212356.g002

Applications of ANN in health care organizational decision-making: A scoping review

PLOS ONE | https://doi.org/10.1371/journal.pone.0212356 February 19, 2019 8 / 22

explicit reference to ANN being used (91 articles), the application of ANN was not in the con-

text of health care organizational decision-making (68 articles), on basis of study exclusion cri-

teria (53 articles) or the articles were irretrievable (8). In total, 80 articles were used for data

collection. Fig 2 illustrates the overall review process including number of articles excluded at

each stage.

Study characteristics

Publication dates ranged from 1997 to 2018 with the number of studies fluctuating each year

(Fig 3A). Studies were published across 24 countries with the majority of first authors from the

United States (26), the United Kingdom and India (7), Taiwan (6) and Italy (5) (Fig 3B). Fig 3A

and 3B illustrate the number of articles published over the years and across varying countries.

Aim and methodology

Main topics or area of interest based on the article’s overall purpose included Organizational

Behaviour (18%), Cardiovascular (14%), Infectious Disease and Telemedicine (7%) (Table 2).

Topics categorized under ‘Organizational Behaviour’ include: behaviour and perspectives, cri-

sis or risk management, clinical and non-clinical decision-making, and resource management

(S2 Appendix). Table 2 lists the main topic areas of articles reviewed.

Applications of ANN were mainly found to be classification (22), prediction (14), and diag-

nosis (10) (Fig 4). Examples of applications include classification of data in medical databases

(i.e. organizing or distinguishing data by relevant categories or concepts) [39], using a hybrid

learning approach for automatic tissue recognition in wound images for accurate wound eval-

uations [40], and comparison of soft-computing techniques for diagnosis of heart conditions

by processing digitally recorded heart sound signals to extract time and frequency features

related to normal and abnormal heart conditions [41]. Applications for prediction included

developing a risk advisor model to predict the chances of diabetes complication according to

changes in risk factors [42], identifying the optimal subset of attributes from a given set of

attributes for diagnosis of heart disease [43], modelling daily patient arrivals in the Emergency

Department [44]. ANN was applied for diagnosis of disease based on age, sex, body mass

index, average blood pressure and blood serum measurements [45], comparing predictive

accuracies of different types of ANN and statistical models for diagnosis of coronary artery dis-

ease [46], diagnosis and risk group assignment for pulmonary tuberculosis among hospitalized

patients [47], and non-invasive diagnosis of early risk in dengue patients [48]. Other examples

include exploring the potential use of mobile phones as a health promotional tool by tracking

daily exercise activities of people and using ANN to estimate a user’s movement[49], or using

ANN to identify factors related to treatment and outcomes potentially impacting patient

length of stay[50]. In addition to S2 Appendix, Fig 4 illustrates the various applications of

ANN identified in the literature review.

With respect to nomenclature or taxonomy, authors mostly reported using artificial neural

networks (36 articles), feed-forward networks (25 articles), a hybrid model (23 articles), recur-

rent feedback networks (6 articles) or other (3 articles) (S2 Appendix). Various types of data

(e.g. patients, cases, images, and signals) and sample sizes were used. Training/testing sets

were in ratios of 50:50, 70:30 or 90:10 and the reported accuracy ranged between 50% and

100%.

Context and key findings

ANN was primarily applied to organizational decision-making at a micro-level (61 articles)

between patients and health care providers in addition to meso-, macro-levels out of which 48

Applications of ANN in health care organizational decision-making: A scoping review

PLOS ONE | https://doi.org/10.1371/journal.pone.0212356 February 19, 2019 9 / 22

articles referenced to micro-level decision-making only; with 29 articles referencing meso-

level applications between patients, health care providers, hospital managers and decision-

makers, out of which 10 referenced meso- only. A small portion (10) of studies applied ANN

at a macro level of decision-making mainly between policy and decision-makers across multi-

ple facilities or health care systems, out of which 2 referenced macro- only. Micro-level appli-

cations of ANN include diagnosis of pulmonary tuberculosis among hospitalized patients by

Fig 3. Article characteristics. (A) Number of articles by publication year. (B) Number of articles by country.

https://doi.org/10.1371/journal.pone.0212356.g003

Applications of ANN in health care organizational decision-making: A scoping review

PLOS ONE | https://doi.org/10.1371/journal.pone.0212356 February 19, 2019 10 / 22

health care providers using models developed for classification and risk group assignment

[47], classify Crohn’s Disease medical images [51], analyse recorded ECG signals to trigger an

alarm for patients and allow collection and transmission of patient information to health care

providers[52]. Meso-level applications include decision-making among managers involving

classification of cost [53], developing a forecasting model to support health care management

decision-making[54], among patients, providers, and hospital managers in order to evaluate

the effect of hospital employee motivation on patient satisfaction [55], and predicting the

adoption of radio frequency identification (RFID) technology adoption in clinical setting [56].

Macro-level applications of ANN include risk-adjustment models for policy-makers of Tai-

wan’s National Health Insurance program [57], a global comparison of the perception of

Table 2. Study areas identified in the review.

Study Area Number of Articles

Organizational Behaviour 18

Other� 15

Cardiovascular 14

Infectious Disease 7

Telemedicine 7

Finance 5

Trauma 5

Medical Imaging 4

Diabetes 4

Surgery 4

Information Systems 4

�Sub-categories of ‘Other’ articles include: elderly studies, renal disease, medical diagnosis, data mining,

pharmacology, fall detection, disorders (epilepsy or autism).

https://doi.org/10.1371/journal.pone.0212356.t002

Fig 4. Types of applications of artificial neural networks identified in the review.

https://doi.org/10.1371/journal.pone.0212356.g004

Applications of ANN in health care organizational decision-making: A scoping review

PLOS ONE | https://doi.org/10.1371/journal.pone.0212356 February 19, 2019 11 / 22

corruption in the health care sector [58], model revenue generation for decision-makers to

determine best indicators of revenue generation in not-for-profit foundations supporting hos-

pitals of varying sizes [59].

Authors reported neural networks reduced computation time in comparison to conven-

tional planning algorithms [60] thereby enabling users to access model output faster in real-

time, outperforming linear regression models in prediction [44, 56, 61–63] and support vector

machines in classification [64, 65]. Limitations centered around the use of small data sets [42,

53, 66–72], limiting data set to continuous variables [69], inability to examine causal relation-

ships [56] or have the network explain weights applied, appropriateness of decision-making

[71, 73, 74], difficulty in implementation or understanding of the output [75]. ANN were cau-

tioned to be used as a proof of concept rather than a successful prediction model [66].

Discussion

This review provides a comprehensive review of the various applications of artificial neural

networks in health care organizational decision-making. To our knowledge, this is the first

attempt to comprehensively describe the use of ANN in health care, from the time of its origins

to current day use, on all levels of organizational decision-making.

Prior efforts have concentrated on a specific domain or aspect of health care and/or limited

study findings to a period of time. A systematic review on the use of ANN as decision-making

tools in the field of cancer reported trends from 1994–2003 in clinical diagnosis, prognosis and

therapeutic guidance for cancer from1994 to 2003, and suggested the need for rigorous meth-

odologies in using neural networks [19]. Another review reported various applications in areas

of accounting and finance, health and medicine, engineering and marketing, however focused

the review on feed-forward neural networks and statistical techniques used in prediction and

classification problems [20]. Outside of medicine and health care, Wong et al. conducted liter-

ature reviews of ANN used in business (from 1988–1995) [76] and finance (1990–1996) [77],

at that time describing the promise of neural networks for increasing integration with other

existing or developing technologies [76, 77]. Data mining is the mathematical core of a larger

process of knowledge discovery from databases otherwise referred to as the ‘KDD process

[78]. The main activities involved in the KDD process include (i) integration and cleaning, (ii)

selection and transformation, (iii) data mining and (iv) evaluation and interpretation. Data

mining pertains to extraction of significant patterns and knowledge discovery and employs

inferring algorithms, such as ANN, to pre-processed data to complete data mining tasks such

as classification and cluster analysis [79]. Data mining and machine learning have produced

practical applications in areas of analysing medical outcomes, detecting credit card fraud, pre-

dicting customer purchase behaviour or predicting personal interests from internet use [80].

Although limited in scope to the field of infertility, Durairaj & Ranjani (2013) conducted a

comparative study of data mining techniques including ANN, suggesting the promise of com-

bining more than one data mining technique for diagnosing or predicting disease [81].

Due to the primitive nature of computer technology mid-20 th

Century, most of the research

in machine learning was theoretical or based on construction of special purpose systems [18].

We found that application of ANN in health care decision-making began in the late 90’s with

fluctuating use over the years. A number of breakthroughs in the field of computer science and

AI bring insight to reported publication patterns [82]. ANN gained prominence with the pub-

lication of a few seminal works including the publication of the backpropagation learning rule

for multilayered feed-forward neural networks [22]. In 1986, backpropagation was proven as a

general purpose and simple procedure, powerful enough for a multi-layered neural network to

use and construct appropriate internal representations based on incoming data [83]. A few

Applications of ANN in health care organizational decision-making: A scoping review

PLOS ONE | https://doi.org/10.1371/journal.pone.0212356 February 19, 2019 12 / 22

years later, the ability of neural networks to learn any type of function was demonstrated [84],

suggesting capabilities of neural networks as universal approximators [85]. During the 90’s,

most of the research was largely experimental and the need for use of ANN as a widely-used

computer paradigm remained warranted [18].

With the digitization of health care [86], hospitals are increasingly able to collect large

amounts of data managed across large information systems [22]. With its ability to process

large datasets, machine learning technology is well-suited for analysing medical data and pro-

viding effective algorithms [22]. Considering the prevalent use of medical information systems

and medical databases, ANN have found useful applications in biomedical areas in diagnosis

and disease monitoring [87].

Although the backpropagation learning rule enabled the use of neural networks in many

hard medical diagnostic tasks, they have been typically used as black box classifiers lacking the

transparency of generating knowledge as well as the ability to explain decision-making [22].

The lack of transparency or interpretability of neural networks continues to be an important

problem since health care providers are often unwilling to accept machine recommendations

without clarity regarding the underlying rationale [88]. Prior to 2006, application of neural

networks included processing of biomedical signals, for example image and speech processing

[89, 90], clinical diagnosis, image analysis and interpretation, and drug development [87]. In

2006, a critical paper described the ability of a neural network to learn faster [91]. Six years

later, the largest deep neural network to date (i.e. depth pertaining to layers of the network),

was trained to classify 1.2 million images in record-breaking time as part of the ImageNet

Large Scale Visual Recognition Challenge [92].

The most successful applications of ANN are found in extremely complex medical situa-

tions [13]. We found ANN to be mainly used for classification, prediction and clinical diagno-

sis in areas of cardiovascular, telemedicine and organizational behaviour. Use of ANN applies

to four general areas of cardiovascular medicine: diagnosis and treatment of coronary artery

disease, general interpretation of electrocardiography, cardiac image analysis and cardiovascu-

lar drug dosing [93]. Telemedicine offers health care providers elaborate solutions for remote

monitoring designed to prevent, diagnose, manage disease and treatment [94] and can include

machine learning techniques to predict clinical parameters such as blood pressure [95]. Pre-

liminary diagnosis of high-risk patients (for disease or attributes) using neural networks pro-

vide hospital administrators with a cost-effective tool in time and resource management [16].

Neural networks have been used effectively as a tool in complex decision-making in strate-

gic management, specifically in strategic planning and performance, assessing decision-mak-

ing [96]. In health care, neural network models have been successfully used to predict quality

determinants (responsiveness, security, efficiency) influencing adoption of e-government ser-

vices [97]. With its ability to discover hidden knowledge and values, scholars have suggested

using ANN to improve care performance and facilitate the adoption of ‘Lean thinking’ or

value-based decision making in health care [87]. An example of ANN facilitating Lean think-

ing adoption in health care contexts is its application to describe ‘information flow’ among

cancer patients by modeling the relationship between quality of life evaluations made by

patients, pharmacists and nurses [87]. ‘Flow’ is a key concept in a Lean System and ‘informa-

tion flow’ is an essential improvement target to the successful operation of a health care system

using a Lean approach [87]. Key success factors or differentiators that define effective machine

learning technology in health care include access to extensive data sources, ease of implemen-

tation, interpretability and buy-in as well as conformance with privacy standards [9]. Support

vector machines are used to model high-dimensional data and are considered state-of-the-art

solutions to problems otherwise not amenable to traditional statistical analysis. Despite its ana-

lytic capabilities, wide-scale adoption remains a challenge, mainly due to methodological

Applications of ANN in health care organizational decision-making: A scoping review

PLOS ONE | https://doi.org/10.1371/journal.pone.0212356 February 19, 2019 13 / 22

complexities and scalability challenges [98]. For example, a systematic review of deep learning

models using electronic health record data recently identified challenges related to the tempo-

rality (e.g. hidden relationships among clinical variables occurring at short and long term

events) and irregularity of information used which can reduce model performance if not han-

dled appropriately [88]. Poor interpretability remains a signicant challenge with implementing

ANN in health care [90]. Zhang et al (2018) report that in comparison to linear models, ANN

are not only difficult to interpret but the identification of predictors (input features) important

for the model also seem to be a challenge [99]. Fisher et al (2016) developed an ANN based

monitoring method evaluating Parkinson’s disease motor symptoms and reported signiciant

challenges with detecting disease states due to the inherent subjectivity underlying the inter-

pretation of disease state descriptors (i.e. the degree of motor symptoms experienced by each

patient would likely vary) [100]. Despite the evident progress in certain areas (e.g. knowledge

and temporal representation, machine learning), the adoption of key standards required for

integration and knowledge sharing (e.g. controlled terminologies, semantic structuring, stan-

dards representing clinical decision logic) has been slow [101] Patel et al. (2009) suggest barri-

ers to progress are related to political, fiscal or cultural reasons and not purely technical. A

national study on the implementation of Health Information Technology (HIT) in the United

States reported a poor understanding of IT staff, informaticians, health information managers

and others playing a significant role in implementation of HIT in health care [102] Barriers to

adoption of HIT include mismatch of return on investment, challenges to workflow in clinical

settings, lack of standards and interoperability, and concerns about privacy and confidentiality

[102].

We found that researchers often adopted a hybrid approach when using neural networks.

Hybrid approaches (e.g. combining two or more techniques/soft-computing paradigms) are

effective in reducing challenges with neural networks when introducing new items to the sys-

tem or having insufficient data [103]. ANN learn (supervised, unsupervised or reinforcement)

based on the iterative adjustment of connection weights using optimization algorithms such as

the backpropagation rule. Challenges related to such algorithms include the necessity of a pre-

viously defined architecture for the model, sensitivity to the initial conditions used in training

[104]. A hybrid model of an ANN and decision tree classifier has been used to predict univer-

sity admissions using data related to student academic merits, background and university

admission criteria. Reported advantages of using a hybrid model included higher prediction

accuracy rates (error rate of <2%), flexibility and faster performance (0.1 second) in compari-

son with a model using neural networks only (20 minutes learning time). Another advantage

reported was improved generalizability, e.g. ability to understand rules extracted that can be

later coded into another type of system [105] Literature suggests extensive use of ANN in busi-

ness applications in particular areas related to financial distress and bankruptcy problems,

stock price forecasting and decision support [106] Hybrid networks have also been developed

in business applications to improve performance of standard models [106]. The integration of

ANN with secondary AI and meta-heuristic methods such as fuzzy logic, genetic, bee colony

algorithms, or artificial immune systems have been proposed to reduce or eliminate challenges

related to ANN (e.g. selection of network topology, initial weights, choice of control parame-

ters) [106]. Applications of hybrid intelligent systems include robotics, medical diagnosis,

speech/natural language understanding, monitoring of manufacturing processes.

Our findings suggest a possible correlation between advancements made in the field of

ANN and publication rates related to the application of ANN in health care organizational

decision-making. Despite the variety of study contexts and applications, ANN continues to be

mainly used for classification, prediction and diagnosis. As suggested by the literature, the

most commonly used taxonomy of ANN found was the feed-forward neural network.

Applications of ANN in health care organizational decision-making: A scoping review

PLOS ONE | https://doi.org/10.1371/journal.pone.0212356 February 19, 2019 14 / 22

However, our study showed a significant use of hybrid models. ANN’s application to facilitate

more micro- and meso-level decision-making compared to macro-level may be explained by

the type and volume of data required and available to build an effective model.

Strengths and limitations

A primary strength of this review is its comprehensive scope and search strategy involving

multiple databases. Variables selected for data collection were based on bodies of work with

similar inquiry and well aligned with the methods of a scoping review. The complex nature of

artificial neural networks required a fundamental understanding for the authors who were

otherwise novice to the field. Studies included in this review did not always use standardized

reporting measures and may include publications of lower quality.

Implications

Practical implications

Current and anticipated advancements in the field of AI will play an influential role in decision-

making related to adopting novel and innovative machine learning based techniques in health

care. Clinical applications of AI include analysis of electronic health records, medical image pro-

cessing, physician and hospital error reduction [107] AI applications in workflow optimization

include payer claim processing, network coordination, staff management, training and educa-

tion, supply costs and management [107] For example, the top three applications of greatest

near-term value (based on the impact of application, likelihood of adoption and value to health

economy) are reported to be robot-assisted surgery (valued at $40 B), virtual nursing assistants

($20B) and administrative workflow assistance ($18 B) [108]. Applications with lowest esti-

mated potential value include preliminary diagnosis ($5B), automated image ($3B) and cyber-

security ($2B) [108]. Our findings warrant the understanding of perspectives and beliefs of

those adopting ANN-based solutions in clinical and non-clinical decision-making.

Patients and families are accessing health information in real-time with the array of AI or

ANN based health care solutions available to them in an open and unstructured market. Clini-

cal applications of ANN-based solutions can have implications on the changing role of health

care providers as well team dynamics and patterns in workflow. The changing role of the phy-

sicians has been at the forefront of recent debates on AI, with some anticipating the positive

impacts of augmenting clinical service with AI based technologies, e.g., enabling early diagno-

sis, or improving understanding of a patient’s medical history with genetic sequencing [109].

Literature suggests a need for bridging disciplines in order to enable of clinicians to benefit

from rapid advancements in technology [101] In addition to the implications for clinical deci-

sion-making, interprofessional team dynamics and processes can be expected to change. For

example, a US based hospital has collaborated with a game development company to create a

virtual world in which surgeons are guided through scenarios in the operating room using

rules, conditions and scripts to practice making decisions, team communication, and leader-

ship [110].

As policy-makers adopt strategies towards a value-based, patient-centred model of care

delivery, decision-makers are required to consider the readiness of health care organizations

for successful implementation and wide-scale adoption of AI or ANN based decision-support

tools. Factors such as easier integration with hospital workflows, patient-centric treatment

plans leading to improved patient outcomes, elimination of unnecessary hospital procedures

and reduced treatment costs can influence wider adoption of AI-based solutions in the health

care industry [107]. Challenges in uptake include the current inability of AI-based solutions to

read unstructured data, the perspectives of health care providers using AI-based solutions, and

Applications of ANN in health care organizational decision-making: A scoping review

PLOS ONE | https://doi.org/10.1371/journal.pone.0212356 February 19, 2019 15 / 22

the lack of supportive infrastructure required for wide-scale implementation [107]. For

improved organizational readiness, the governance and operating model of health care organi-

zations need to enable a workforce and culture that will support the use of AI to enhance effi-

ciency, quality and patient outcomes [108].

Machine learning from unstructured data (e.g. patient health records, photos, reviews,

social media data from mobile applications and devices) remain a critical unmet need for hos-

pitals [107, 111]. Currently, most of the data in health care is unstructured and difficult to

share [107] Wide-scale implementation and adoption of AI service solutions requires strong

partnerships between AI technology vendors and health care organizations [107]. Policies

encouraging transparency and sharing of core datasets across public and private sectors can

stimulate higher levels of innovation-oriented competition and research productivity [112].

Theoretical implications

Several theoretical implications emerge from our study findings. Healthcare organizations are

complex adaptive systems embedded in larger complex adaptive systems[113]; health care

organizational decision-making can appropriately rely on ANN as an internalized rule set. The

change of health care delivery from single to multiple settings and providers has led to new

complexities around how health care delivery needs are being structured and managed (e.g.,

support required for delivering collaborative care or patient participatory medicine) [1]. Tradi-

tional decision-making processes based on stable and predictable systems are no longer rele-

vant, due to the complex and emergent nature of contemporary health care delivery systems

[1]. Yet the health care organizational decision-making literature suggests the focus of deci-

sion-making persistently remains on problems that are visible, while the larger system within

which health care delivery organizations exist remains unacknowledged [1]. Using complex

adaptive systems (CAS) theory to understand the functionality of AI can provide critical

insights: first, AI enhances adaptability to change by strengthening communication among

agents, which in turn fosters rapid collective response to change, and further, AI possesses the

potential to generate a collective memory for social systems within an organization [114].

The theory of CAS has been used as an alternative approach to improve our understanding

and scaling up of health services; CAS theory shifts decision-making towards embracing

uncertainty, non-linear processes, varying context and emergent characteristics [115]. Interde-

pendent organizational factors such as clinical practice, organization, information manage-

ment research education and professional development, are built around multiple self-

adjusting interacting systems [116]. Agents (e.g. users of the system) respond to their environ-

ment based on internalized rule sets that are not necessarily explicit, shared or need to be

understood by another agent [116]. Although lacking the ability to explain decision-making,

ANN-based decision-support tools enable health care organizational decision-makers to

respond to complex and emergent environments using incoming and evolving data.

Conclusion

Our study found artificial neural networks can be applied across all levels of health care organi-

zational decision-making. Influenced by advancements in the field, decision-makers are taking

advantage of hybrid models of neural networks in efforts to tailor solutions to a given problem.

We found ANN-based solutions applied on the meso- and macro-level of decision-making

suggesting the promise of its use in contexts involving complex, unstructured or limited infor-

mation. Successful implementation and adoption may require an improved understanding of

the ethical, societal, and economic implications of applying ANN in health care organizational

decision-making.

Applications of ANN in health care organizational decision-making: A scoping review

PLOS ONE | https://doi.org/10.1371/journal.pone.0212356 February 19, 2019 16 / 22

Supporting information

S1 Checklist. Preferred Reporting Items for Systematic Reviews and Meta-Analyses

(PRISMA) checklist.

(PDF)

S1 Appendix. Search strategy and syntax.

(PDF)

S2 Appendix. Summary of findings.

(PDF)

S3 Appendix. Glossary of terms.

(PDF)

S1 Workflow. Preferred Reporting Items for Systematic Reviews and Meta-Analyses

(PRISMA) flowchart.

(PDF)

Author Contributions

Conceptualization: Nida Shahid, Whitney Berta.

Data curation: Nida Shahid.

Formal analysis: Nida Shahid, Tim Rappon, Whitney Berta.

Investigation: Nida Shahid, Tim Rappon, Whitney Berta.

Methodology: Nida Shahid, Tim Rappon, Whitney Berta.

Project administration: Nida Shahid.

Supervision: Whitney Berta.

Writing – original draft: Nida Shahid.

Writing – review & editing: Nida Shahid, Tim Rappon, Whitney Berta.

References 1. Kuziemsky C. Decision-making in healthcare as a complex adaptive system. Healthc Manage Forum.

2016; 29(1):4–7. https://doi.org/10.1177/0840470415614842 PMID: 26656389.

2. Deloitte. Global health care outlook: The evolution of smart health care. 2018.

3. Young DW, Ballarin E. Strategic decision-making in healthcare organizations: it is time to get serious.

Int J Health Plann Manage. 2006; 21(3):173–91. PMID: 17044545.

4. Singh JV. Slack, and Risk Taking in Organizational Decision-Making The Academy of Management

Journal. 1986; 29(3):562–85.

5. French R, Rayner C, G R, Rumbles S, Schermerhorn J, Hunt J, et al. Organizational Behaviour. 2nd

ed. New York: John Wiley & Sons; 2011.

6. Awowale A. Decision Making in Healthcare Systems: Roles and Responsibilities: University of Mary-

land; 2017.

7. IBM Corporation. Data-driven healthcare organizations use big data analytics for big gains. Somers,

NY: IBM Corporation, 2013.

8. Grand View Research. Healthcare Predictive Analytics Market Analysis By Application (Operations

Management, Financial, Population Health, Clinical), By End-Use (Payers, Providers), By Region

(North America, Europe, Asia Pacific, Latin America, MEA) And Segment Forecasts, 2018–2025.

2016 November 2016. Report No.

Applications of ANN in health care organizational decision-making: A scoping review

PLOS ONE | https://doi.org/10.1371/journal.pone.0212356 February 19, 2019 17 / 22

9. Tatcher L. The Dangers of Commoditized Machine Learning in Healthcare: 5 Key Differentiators that

Lead to Success: Healthcare.ai; 2018 [cited 2018]. Available from: https://healthcare.ai/dangers-of-

commoditized-machine-learning-in-healthcare/.

10. Jiang F, Jiang Y, Zhi H, Dong Y, Li H, Ma S, et al. Artificial intelligence in healthcare: past, present and

future. Stroke Vasc Neurol. 2017; 2(4):230–43. https://doi.org/10.1136/svn-2017-000101 PMID:

29507784; PubMed Central PMCID: PMCPMC5829945.

11. Lee CW, Park JA. Assessment of HIV/AIDS-related health performance using an artificial neural net-

work. Information & Management. 2001; 38(4):231–8. https://doi.org/10.1016/S0378-7206(00)00068-

9 PubMed PMID: WOS:000165979600003.

12. Sordo M. Introduction to Neural Networks in Healthcare. 2002.

13. Bartosch-Harlid A, Andersson B, Aho U, Nilsson J, Andersson R. Artificial neural networks in pancre-

atic disease. Br J Surg. 2008; 95(7):817–26. https://doi.org/10.1002/bjs.6239 PMID: 18551536.

14. Goss EP, Vozikis GS. Improving Health Care Organizational Management Through Neural Network

Learning. Health Care Management Science. 2002; 5(3):221–7. PMID: 12363049

15. Kaur H, Wasan SK. Empirical Study on Applications of Data Mining Techniques in Healthcare. Journal

of Computer Science. 2006; 2(2):194–200.

16. Nolting J. Developing a neural network model for health care. AMIA Annu Symp Proc. 2006: 1049.

PMID: 17238669; PubMed Central PMCID: PMCPMC1839654.

17. Somers J. Is AI Riding the One-Trick Pony? MIT Technology Review. 2017.

18. Schocken S, Ariav G. Neural Networks for Decision Support Systems: Problems and Opportunities.

New York: Center for Research on Information Systems, Stern School of Business, New York Univer-

sity, 1991.

19. Lisboa PJ, Taktak AF. The use of artificial neural networks in decision support in cancer: a systematic

review. Neural Netw. 2006; 19(4):408–15. https://doi.org/10.1016/j.neunet.2005.10.007 PMID: 16483741.

20. Paliwal M, Kumar U. Neural networks and statistical techniques: A review of applications. Expert Sys-

tems with Applications. 2009; 36(1):2–17.

21. Guler S, Hurton S, C winn M, Molinari M. Levels in Decision Making and Techniques for Clinicians.

International Journal of Digestive Diseases. 2015; 1(1:2).

22. Kononenko I. Machine learning for medical diagnosis: history, state of the art and perspective. Artif

Intell Med. 2001; 23(1):89–109. PMID: 11470218.

23. Scarborough D, Somers MJ. Neural Networks in Organizational Research: Applying Pattern Recogni-

tion to the Analysis of Organizational Behaviour. Washington, D.C.: American Psychological Associa-

tion; 2006.

24. Sarle W, editor Neural Networks and Statistical Models. Nineteenth Annual SAS Users Group INterna-

tional Conference; 1994; Cary, NC, USA.

25. Kayri M. Data Optimization with Multilayer Perceptron Neural Network and Using New Pattern in Deci-

sion Tree Comparitively. Journal of Computer Science. 2010; 6(5):606–12.

26. Agatonovic-Kustrin S, Beresford R. Basic concepts of artificial neural network (ANN) modeling and its

application in pharmaceutical research. J Pharm Biomed Anal. 2000; 22(5):717–27. PMID: 10815714.

27. da Silva IN, Hernane Spatti S, Andrade Flauzino R. Chapter 2: Artificial Neural Network Architectures

and Training Processes. Artificial Neural Networks: A Practical Course. Switzerland: Springer Inter-

national Publishing 2017.

28. Penny W, Frost D. Neural Networks in Clinical Medicine. Medical Decision Making. 1996; 16:386–98.

https://doi.org/10.1177/0272989X9601600409 PMID: 8912300

29. Forrstrom J, Dalton KJ. Artificial Neural Networks for Decision Support in Clinical Medicine. Annals of

Medicine. 1995; 27(5):509–17. PMID: 8541025

30. Sharma A, Chopra A. Artificial Neural Networks: Applications in Management. Journal of Business

and Management. 2013; 12(5):32–40.

31. Tu JV. Advantages and disadvantages of using artificial neural networks versus logistic regression for

predicting medical outcomes. J Clin Epidemiol. 1996; 49(11):1225–31. PMID: 8892489.

32. Han J, Pei J, Kamber M. Data Mining: Concepts and Techniques. Waltham, MA: Elsevier Inc.; 2012.

33. Picard RR, Ber KN. Data Splitting. The American Statistician 2010; 44(2):140–7.

34. Reitermanova Z. Data Splitting. WDS’10 Proceedings of Contributed Papers. 2010;Part 1:31–6. 978-

80-7378-139-2.

35. May RJ, Maier HR, Dandy GC. Data splitting for artificial neural networks using SOM-based stratified

sampling. Neural Networks. 2010; 23:283–94. https://doi.org/10.1016/j.neunet.2009.11.009 PMID:

19959327

Applications of ANN in health care organizational decision-making: A scoping review

PLOS ONE | https://doi.org/10.1371/journal.pone.0212356 February 19, 2019 18 / 22

36. Amazon. Amazon Machine Learning: Developer Guide. 2018.

37. McGarry K, Wermster S, MacIntyre J. Hybrid Neural Systems: From SImple Coupling to Fully Inte-

grated Nueral Networks. Neural Computing Surveys 2. 1999:62–93.

38. Tickle AB, Andrews R, Golea M, Diederich J. The truth will come to light: directions and challenges in

extracting the knowledge embedded within trained artificial neural networks. IEEE Trans Neural Netw.

1998; 9(6):1057–68. https://doi.org/10.1109/72.728352 PMID: 18255792.

39. Verma B, Hassan SZ. Hybrid ensemble approach for classification. Applied Intelligence. 2011; 34

(2):258–78. http://dx.doi.org/10.1007/s10489-009-0194-7. PMID: 856176289.

40. Veredas FJ, Mesa H, Morente L. A hybrid learning approach to tissue recognition in wound images.

International Journal of Intelligent Computing and Cybernetics. 2009; 2(2):327–47. http://dx.doi.org/

10.1108/17563780910959929. PubMed PMID: 228078109.

41. Samanta B, Nataraj C. Automated diagnosis of cardiac state in healthcare systems using computa-

tional intelligence. International Journal of Services Operations and Informatics. 2008; 3(2):162–77.

http://dx.doi.org/10.1504/IJSOI.2008.019331. PMID: 34063662; 200809-B7-0052794 (MB); 200809-

51-0177345 (CI).

42. Sangi M, Win KT, Shirvani F, Mohammad-Reza N- R, Shukla N. Applying a Novel Combination of

Techniques to Develop a Predictive Model for Diabetes Complications. PLoS One. 2015; 10(4). http://

dx.doi.org/10.1371/journal.pone.0121569. PubMed PMID: 1691593088.

43. Vivekanandan T, Iyengar NCSN. Optimal feature selection using a modified differential evolution algo-

rithm and its effectiveness for prediction of heart disease. Computers in Biology and Medicine. 2017;

90:125–36. http://dx.doi.org/10.1016/j.compbiomed.2017.09.011. PubMed PMID: 1954368817.

https://doi.org/10.1016/j.compbiomed.2017.09.011 PMID: 28987988

44. Xu M, Wong TC, Chin KS. Modeling daily patient arrivals at Emergency Department and quantifying

the relative importance of contributing variables using artificial neural network. Decision Support Sys-

tems. 2013; 54(3):1488. PubMed PMID: 1313261404.

45. Olatubosun O, Olusoga F, Abayomi F. Diabetes Diagnosis with Maximum Covariance Weighted Resil-

ience Back Propagation Procedure. International Journal of Health and Economic Development.

2015; 1(2):1–13. PubMed PMID: 1690740202.

46. Süt N, Senocak M. Assessment of the performances of multilayer perceptron neural networks in com-

parison with recurrent neural networks and two statistical methods for diagnosing coronary artery dis-

ease. Expert Systems. 2007; 24(3):131. http://dx.doi.org/10.1111/j.1468-0394.2007.00425.x. PMID:

196936703.

47. Aguiar FS, Torres RC, Pinto JV, Kritski AL, Seixas JM, Mello FC. Development of two artificial neural

network models to support the diagnosis of pulmonary tuberculosis in hospitalized patients in Rio de

Janeiro, Brazil. Medical and Biological Engineering and Computing. 2016; 54(11):1751–9. http://dx.

doi.org/10.1007/s11517-016-1465-1. PMID: 27016365.

48. Ibrahim F, Faisal T, Mohamad Salim MI, Taib MN. Non-invasive diagnosis of risk in dengue patients

using bioelectrical impedance analysis and artificial neural network. Medical and Biological Engineer-

ing and Computing. 2010; 48(11):1141–8. http://dx.doi.org/10.1007/s11517-010-0669-z. PubMed

PMID: 776582910; https://doi.org/10.1007/s11517-010-0669-z PMID: 20683676.

49. Anderson I, Maitland J, Sherwood S, Barkhuss L, Chalmers M, Hall M, et al. Shakra: Tracking and

Sharing Daily Activity Levels with Unaugmented Mobile Phones. Mobile Networks and Applications.

2007; 12(2–3):185–99.

50. Kudyba S, Gregorio T. Identifying factors that impact patient length of stay metrics for healthcare pro-

viders with advanced analytics. Health Informatics J. 2010; 16(4):235–45. https://doi.org/10.1177/

1460458210380529 PMID: 21216804.

51. Ahmed SS, Dey N, Ashour AS, Sifaki-pistolla D, Balas-timar D, Balas VE, et al. Effect of fuzzy parti-

tioning in Crohn’s disease classification: a neuro-fuzzy-based approach. Medical and Biological Engi-

neering and Computing. 2017; 55(1):101–15. http://dx.doi.org/10.1007/s11517-016-1508-7. PubMed

PMID: 1856582777. https://doi.org/10.1007/s11517-016-1508-7 PMID: 27106754

52. Atoui H, Telisson D, Fayn J, Rubel P. Ambient Intelligence and Pervasive Architecture Designed within

the EPI-MEDICS Personal ECG Monitor. International Journal of Healthcare Information Systems and

Informatics. 2008; 3(4):69–80. PMID: 33773373; 200809-B7-0052020 (MB); 200809-82-0176356

(CI).

53. Berardi VL, Zhang GP. The effect of misclassification costs on neural network classifiers. Decision Sci-

ences. 1999; 30(3):659–82. PMID: 198182834.

54. Eswaran C, Logeswaran R. A dual hybrid forecasting model for support of decision making in health-

care management. Advances in Engineering Software. 2012; 53:23–32. http://dx.doi.org/10.1016/j.

advengsoft.2012.07.006. PubMed PMID: 1770299318; PQ0002257828.

Applications of ANN in health care organizational decision-making: A scoping review

PLOS ONE | https://doi.org/10.1371/journal.pone.0212356 February 19, 2019 19 / 22

55. Hanafizadeh P, Paydar NR, Aliabadi N. Neural Network-based Evaluation of the Effect of the Motiva-

tion of Hospital Employees on Patients’ Satisfaction. International Journal of Healthcare Information

Systems and Informatics. 2010; 5(4):1–19. http://dx.doi.org/10.4018/jhisi.2010100101. PubMed

PMID: 1671366574; 16764023.

56. Chong AY- L, Liu MJ, Luo J, Keng-Boon O. Predicting RFID adoption in healthcare supply chain from

the perspectives of users. International Journal of Production Economics. 2015; 159:66–75. http://dx.

doi.org/10.1016/j.ijpe.2014.09.034. PubMed PMID: 1669878031; PQ0001074931.

57. Hsu S, Lin C, Yang Y. INTEGRATING NEURAL NETWORKS FOR RISK-ADJUSTMENT MODELS.

Journal of Risk and Insurance. 2008; 75(3):617–42. PMID: 226937475.

58. Buscema PM, Gitto L, Russo S, Marcellusi A, Fiori F, Maurelli G, et al. The perception of corruption in

health: AutoCM methods for an international comparison. Quality and Quantity. 2017; 51(1):459–77.

http://dx.doi.org/10.1007/s11135-016-0315-4. PubMed PMID: 1857580006.

59. Malliaris ME, Pappas M. Revenue Generation In Hospital Foundations: Neural Network Versus

Regression Model Recommendations. International Journal of Management and Information Sys-

tems. 2011; 15(1):59–66. PubMed PMID: 853756956.

60. Bajo J, de Paz JF, de Paz Y, Corchado JM. Integrating case-based planning and RPTW neural net-

works to construct an intelligent environment for health care. Expert Systems with Applications. 2009;

36(3):5844–58. http://dx.doi.org/10.1016/j.eswa.2008.07.029. PMID: 33360064; 200904-65-0021007

(CI).

61. Bourdès V, Ferrières J, Amar J, Amelineau E, Bonnevay S, Berlion M, et al. Prediction of persistence of combined evidence-based cardiovascular medications in patients with acute coronary syndrome

after hospital discharge using neural networks. Medical and Biological Engineering and Computing.

2011; 49(8):947–55. http://dx.doi.org/10.1007/s11517-011-0785-4. PubMed PMID: 878056273;

https://doi.org/10.1007/s11517-011-0785-4 PMID: 21598000.

62. Chesney T, Penny K, Oakley P, Davies S, Chesney D, Maffulli N, et al. Data Mining Medical Informa-

tion: Should Artificial Neural Networks Be Used to Analyse Trauma Audit Data? International Journal

of Healthcare Information Systems and Informatics. 2006; 1(2):51. PMID: 35350817; 200902-B7-

0003850 (MB); 200902-82-0010190 (CI).

63. Ernest Preston G, Vozikis GS. Improving Health Care Organizational Management Through Neural

Network Learning. Health Care Management Science. 2002; 5(3):221–7. PMID: 12363049;

12363049.

64. Chao C-M, Yu C-T, Cheng B-W. DATA MINING MODELS FOR DIAGNOSING ACUTE RENAL FAIL-

URE. International Journal of Organizational Innovation (Online). 2012; 4(4):155–74. PubMed PMID:

1011813323.

65. Oliva JT, Garcia Rosa JL. How an epileptic EEG segment, used as reference, can influence a cross-

correlation classifier? Applied Intelligence. 2017; 47(1):178–96. http://dx.doi.org/10.1007/s10489-016-

0891-y. PubMed PMID: 1906055411.

66. Belmonte M. Prediction of attention in autism from single-trial EEG using artificial neural networks.

SIGBIO Newsl. 1997; 17(2):2–15. https://doi.org/10.1145/270572.270573

67. Emanet N, Öz HR, Bayram N, Delen D. A comparative analysis of machine learning methods for clas-

sification type decision problems in healthcare. Decision Analytics. 2014; 1(1):1–20. http://dx.doi.org/

10.1186/2193-8636-1-6. PubMed PMID: 1652776285.

68. Grossi E, Compare A, Buscema M. The concept of individual semantic maps in clinical psychology: a

feasibility study on a new paradigm. Quality and Quantity. 2014; 48(1):15–35. http://dx.doi.org/10.

1007/s11135-012-9746-8. PubMed PMID: 1473700449.

69. Lin C-f, Leboulluec AK, Zeng L, Chen VC, Gatchel RJ. A decision-making framework for adaptive pain

management. Health Care Management Science. 2014; 17(3):270–83. http://dx.doi.org/10.1007/

s10729-013-9252-0. PubMed PMID: 1555370084; https://doi.org/10.1007/s10729-013-9252-0 PMID:

23974825.

70. Liu NT, Holcomb JB, Wade CE, Batchinsky AI, Cancio LC, Darrah MI, et al. Development and valida-

tion of a machine learning algorithm and hybrid system to predict the need for life-saving interventions

in trauma patients. Medical and Biological Engineering and Computing. 2014; 52(2):193–203. http://

dx.doi.org/10.1007/s11517-013-1130-x. PubMed PMID: 1491042940; https://doi.org/10.1007/

s11517-013-1130-x PMID: 24263362.

71. Oztekin A. An Analytical Approach to Predict the Performance of Thoracic Transplantations. Journal

of Centrum Cathedra. 2012; 5(2):185–206. PubMed PMID: 1082041429.

72. Zhang S, Tjortjis C, Zeng X, Qiao H, Buchan I, Keane J. Comparing data mining methods with logistic

regression in childhood obesity prediction. Information Systems Frontiers. 2009; 11(4):449–60. http://

dx.doi.org/10.1007/s10796-009-9157-0. PubMed PMID: 232058422.

Applications of ANN in health care organizational decision-making: A scoping review

PLOS ONE | https://doi.org/10.1371/journal.pone.0212356 February 19, 2019 20 / 22

73. Goodey RD, Brickley MR, Hill CM, Shepherd JP. A controlled trial of three referral methods for patients

with third molars. British Dental Journal. 2000; 189(10):556–60. PMID: 11128259

74. Sheikhtaheri A, Sadoughi F, Hashemi Dehaghi Z. Developing and Using Expert Systems and Neural

Networks in Medicine: A Review on Benefits and Challenges. Journal of Medical Systems. 2014; 38

(9):1–110. http://dx.doi.org/10.1007/s10916-014-0110-5. PubMed PMID: 1551595080; https://doi.org/

10.1007/s10916-014-0110-5 PMID: 25027017.

75. Houston AL, Chen H, Hubbard SM, Schatz BR, Ng TD, Sewell RR, et al. Medical Data Mining on the

Internet: Research on a Cancer Information System. The Artificial Intelligence Review. 1999; 13(5–

6):437–66. http://dx.doi.org/10.1023/A:1006548623067. PMID: 198128879.

76. Wong BK, Bondnovich T, Selvi Y. Neural network applications in business: A review and analysis of

the literature (1988–1975). Decision Support Systems. 1997; 19(4):301–20.

77. Wong BK, Selvib Y. Neural network applications in finance: A review and analysis of literature (1990–

1996). Information & Management. 1998; 34(3):129-.

78. Maimon OZ, Rokach L. Data Mining and Knowledge Discovery Handbook. Second ed. New York:

Springer; 2010 2010.

79. Stahl F, Jordanov I. An overview of the use of neural networks for data mining tasks. WIREs Data Min-

ing Knowledge Discovery. 2012; 2:193–208. https://doi.org/10.1002/widm.1052

80. Mitchell TM. Machine Learning and Data Mining. Communications of the ACM. 1999; 42(11).

81. Durairaj M, Ranjani V. Data Mining Applications In Healthcare Sector: A Study. International Journal of

Scientific & Technology Research. 2013; 2(10).

82. McCulloch WS, Pitts W. A logical calculus of the ideas immanent in nervous activity. Bulletin of Mathe-

matical Biology. 1943; 52(1/2):99–115; discussion 73–97. PMID: 2185863.

83. Rumelhart DE, Hinton GE, Williams RJ. Learning representations by back-propagating errors. Nature.

1986;323.

84. Cybenko G. Approximation by Superpositions of a Sigmoidal Function. Mathematics of Control, Sig-

nals and Systems. 1989; 2:303–14.

85. Hornik K. Approximation Capabilities of Multilayer Feedforward Networks. Neural Networks 1990;

4:251–7.

86. Anderson CL, Agarwal R. The Digitization of Healthcare: Boundary Risks, Emotion, and Consumer

Willingness to Disclose Personal Health Information. Information Systems Research. 2011; 22

(3):469–90.

87. Moghimi FH, Wickramasinghe N. Chapter 2: Artificial Neural Network for Excellence to Facilitate Lean

Thinking Adoption in Healthcare Contexts. Lean Thinking for Healthcare. New York: Springer Sci-

ence; 2014.

88. Xiao C, Choi E, Sun J. Opportunities and challenges in developing deep learning models using elec-

tronic health records data: a systematic review. J Am Med Inform Assoc. 2018; 25(10):1419–28.

https://doi.org/10.1093/jamia/ocy068 PMID: 29893864; PubMed Central PMCID: PMCPMC6188527.

89. Begg R, Kamruzzaman J, Sarker R. Neural Networks in Healthcare: Potential and Challenges. Her-

shey, United States: Idea Group Publishing; 2006.

90. Ferguson J. Neural Networks in Healthcare 2018 [cited 2018 26 March 2018]. Available from: https://

royaljay.com/healthcare/neural-networks-in-healthcare/.

91. Hinton GE, Osindero S, Teh YW. A fast learning algorithm for deep belief nets. Neural Comput. 2006;

18(7):1527–54. https://doi.org/10.1162/neco.2006.18.7.1527 PMID: 16764513.

92. Krizhevsky A, Sutskever I, Hinton GE. ImageNet Classification with Deep Convolutional Neural Net-

works. 2012.

93. Itchhaporia D, Snow PB, Almassy RJ, Oetgen WJ. Artificial neural networks: current status in cardio-

vascular medicine. J Am Coll Cardiol. 1996; 28(2):515–21. https://doi.org/10.1016/0735-1097(96)

00174-X PMID: 8800133.

94. Lymberis A, editor Smart Wearables for Remote Health Monitoring, from Prevention to Rehabilitation:

Current R&D, Future Challenges. 4th Annual IEEE Conference on Information Technology Applica-

tions in Biomedicine; 2003; United Kingdom.

95. Kwong EW, Wu H, Pang GK. A prediction model of blood pressure for telemedicine. Health Informatics

J. 2016. https://doi.org/10.1177/1460458216663025 PMID: 27496863.

96. Sharma AS, Chopra A. Artificial Neural Networks: Applications In Management. Journal of Business

and Management. 2013; 12(5):32–40.

97. Sharma SK, Govindaluri SM, Gattoufi S. Understanding and predicting the quality determinants of e-

government services: A two-staged regression-neural network model. Journal of Modelling in Manage-

ment. 2015; 10(3):325–40.

Applications of ANN in health care organizational decision-making: A scoping review

PLOS ONE | https://doi.org/10.1371/journal.pone.0212356 February 19, 2019 21 / 22

98. Milenova B, Yarmus JS, Campos MM, editors. SVM in Oracle Database 10g: Removing the Barriers

to Widespread Adoption of Support Vector Machines. Proceedings of the 31st VLDB Conference;

2005; Trondheim, Norway.

99. Zhang Z, Beck MW, Winkler DA, Huang B, Sibanda W, Goyal H, et al. Opening the black box of neural

networks: methods for interpreting neural network models in clinical applications. Ann Transl Med.

2018; 6(11):216. https://doi.org/10.21037/atm.2018.05.32 PMID: 30023379; PubMed Central PMCID:

PMCPMC6035992.

100. Fisher JM, Hammerla NY, Ploetz T, Andras P, Rochester L, Walker RW. Unsupervised home monitor-

ing of Parkinson’s disease motor symptoms using body-worn accelerometers. Parkinsonism Relat

Disord. 2016; 33:44–50. https://doi.org/10.1016/j.parkreldis.2016.09.009 PMID: 27637282.

101. Patel VL, Shortliffe EH, Stefanelli M, Szolovits P, Berthold MR, Bellazzi R, et al. The coming of age of

artificial intelligence in medicine. Artif Intell Med. 2009; 46(1):5–17. https://doi.org/10.1016/j.artmed.

2008.07.017 PMID: 18790621; PubMed Central PMCID: PMCPMC2752210.

102. Hersh W, Wright A. What workforce is needed to implement the health information technology

agenda? Analysis from the HIMSS analytics database. AMIA Annu Symp Proc. 2008: 303–7. PMID:

18998990; PubMed Central PMCID: PMCPMC2656033.

103. Abbas A, Zhang L, Khan SU. A survey on context-aware recommender systems based on computa-

tional intelligence techniques. Computing. 2015; 97:667–90.

104. Gutierrez PA, Hervas-Martinez C. Hybrid Artificial Neural Networks: Models, Algorithms and Data.

IWANN Berlin Heidelberg: Springer-Verlag; 2011. p. 177–94.

105. Fong S, Si YW, Biuk-Aghai RP. Applying a Hybrid Model of Neural Network and Decision Tree Classi-

fier for Predicting University Admission. IEEE2009.

106. Tkac M, Verner R. Artificial neural networks in business: Two decades of research. Applied Soft Com-

puting. 2016; 38:788–804.

107. Buttar HS, Rajan V. From $600 M to $6 Billion, Artificial Intelligence Systems Poised for Dramatic Mar-

ket Expansion in Healthcare. Frost & Sullivan, 2016.

108. Collier M, Fu R, Yin L, Christiansen P. Artificial Intelligence: Healthcare’s New Nervous System.

Accenture, 2017.

109. Reller R. AI’s revolutionary role in healthcare 2017 [cited 2017 1 December 2017]. Available from:

https://www.elsevier.com/connect/ais-revolutionary-role-in-healthcare.

110. Smith R. Artificial intelligence: coming soon to a hospital near you: STAT; 2017 [24 April 2018]. Avail-

able from: https://www.statnews.com/2017/04/13/artificial-intelligence-surgeons-hospital/.

111. Murdoch TB, Detsky AS. The inevitable application of big data to health care. JAMA. 2013; 309

(13):1351–2. https://doi.org/10.1001/jama.2013.393 PMID: 23549579.

112. Cockburn IM, Henderson R, Stern S. The Impact of Artificial Intelligence on Innovation: An Exploratory

Analysis. NBER Conference on Research Issues in Artificial Intelligence; Toronto2017.

113. McDaniel RR, Lanham HJ, Anderson RA. Implications of complex adaptive systems theory for the

design of reserch on health care organizations. Health Care Management Rev. 2009; 34(2):191–9.

114. Saadat P. A Complex Adaptive Systems Perspective to Appreciative Inquiry: A Theoretical Analysis.

Advances in Business Research. 2015; 6:1–13.

115. Paina L, Peters DH. Understanding pathways for scaling up health services through the lens of com-

plex adaptive systems. Health Policy Plan. 2012; 27(5):365–73. https://doi.org/10.1093/heapol/czr054

PMID: 21821667.

116. Plsek PE, Greenhalgh T. Complexity science: The challenge of complexity in health care. BMJ. 2001;

323(7313):625–8. PMID: 11557716; PubMed Central PMCID: PMCPMC1121189.

Applications of ANN in health care organizational decision-making: A scoping review

PLOS ONE | https://doi.org/10.1371/journal.pone.0212356 February 19, 2019 22 / 22

Copyright of PLoS ONE is the property of Public Library of Science and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use.