Cost, quality, and access to care have been Americans’ main concerns when it comes to health care.

RESEARCH ARTICLE Open Access

Exploring resistance to implementation of
welfare technology in municipal healthcare
services – a longitudinal case study
Etty R. Nilsen1*, Janne Dugstad2, Hilde Eide2, Monika Knudsen Gullslett2 and Tom Eide2

Abstract

Background: Industrialized and welfare societies are faced with vast challenges in the field of healthcare in the
years to come. New technological opportunities and implementation of welfare technology through co-creation are
considered part of the solution to this challenge. Resistance to new technology and resistance to change is, however,
assumed to rise from employees, care receivers and next of kin. The purpose of this article is to identify and describe
forms of resistance that emerged in five municipalities during a technology implementation project as part of the care
for older people.

Methods: This is a longitudinal, single-embedded case study with elements of action research, following an
implementation of welfare technology in the municipal healthcare services. Participants included staff from the
municipalities, a network of technology developers and a group of researchers. Data from interviews, focus
groups and participatory observation were analysed.

Results: Resistance to co-creation and implementation was found in all groups of stakeholders, mirroring the
complexity of the municipal context. Four main forms of resistance were identified: 1) organizational resistance, 2)
cultural resistance, 3) technological resistance and 4) ethical resistance, each including several subforms. The resistance
emerges from a variety of perceived threats, partly parallel to, partly across the four main forms of resistance, such as
a) threats to stability and predictability (fear of change), b) threats to role and group identity (fear of losing power or
control) and c) threats to basic healthcare values (fear of losing moral or professional integrity).

Conclusion: The study refines the categorization of resistance to the implementation of welfare technology in
healthcare settings. It identifies resistance categories, how resistance changes over time and suggests that resistance
may play a productive role when the implementation is organized as a co-creation process. This indicates that the
importance of organizational translation between professional cultures should not be underestimated, and supports
research indicating that focus on co-initiation in the initial phase of implementation projects may help prevent different
forms of resistance in complex co-creation processes.

Keywords: Ethical resistance, Welfare technology, Innovation, Co-creation, Municipal healthcare

* Correspondence: [email protected]
1The Science Centre Health and Technology, School of Business, University
College of Southeast Norway, Postboks 235, N-3603 Kongsberg, Norway
Full list of author information is available at the end of the article

© The Author(s). 2016 Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0
International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and
reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to
the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver
(http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.

Nilsen et al. BMC Health Services Research (2016) 16:657
DOI 10.1186/s12913-016-1913-5

Background
Healthcare services face vast challenges that will increase
in the years to come, partly due to demographic changes
including ageing populations [1, 2]. Welfare technology
is viewed as one important means to meet these chal-
lenges. Implementation of digital night surveillance tech-
nologies in nursing homes and home care services has
emerged as a potentially efficient way of meeting the need
for monitoring persons for healthcare and safety reasons.
This is an alternative to calling in on, for example, patients
with dementia or intellectual disabilities, and potentially
waking them up at night. However, the application and
use of digital surveillance technologies in the care for vul-
nerable individuals generates considerable ethical debate
[3–5]. Implementation of welfare technology also implies
innovation and organizational change, which is often met
by different kinds of resistance. Resistance can be found
on individual, organizational, and institutional levels, and
these levels are often inter-connected [6–8]. This paper
explores if and how resistance occurs on different levels in
the initial phase of digital surveillance technology imple-
mentation in municipal nursing homes and home care
services.

Implementation of innovation
Innovation has been defined as “the intentional introduc-
tion and application within a role, group, or organization,
of ideas, processes, products or procedures, new to the
relevant unit of adoption, designed to significantly benefit
the individual, the group, or wider society” [9, 10]. This
definition has become widely accepted among researchers
[11, 12]. It captures many aspects of the innovation
process under study, as it aims at implementing new tech-
nologies and developing new ways of working in order to
benefit the individual service user and the healthcare
organization. Implementation is seen as one of the four
stages of innovation: dissemination, adoption, implemen-
tation and continuation [13]. The implementation stage is
according to Rogers “that which occurs when an individ-
ual puts an innovation into use” ([14]:474).
Implementation of technology initiates a change process

and has the potential to alter the way we work, how we
organize work and the power relations in an organization.
However, a large number of change initiatives fail due
to unfocused and insecure management and lack of
systematic project management [15, 16] or are slow to
be implemented (e.g. [17–19]). The implementation phase
is increasingly becoming a phase where the technology
developers and the customers cooperate closely, and in
the business literature it is coined as co-development of
the product [20] or co-creation of value [21]. The concept
of co-creation implies close and continuous interaction
in the implementation phase between the innovators
and developers of the technology and the customers.

The technology developers may lack knowledge about
the market and the users, while customers often also
lack familiarity of technological language and technol-
ogy proficiency. In the implementation phase of, for ex-
ample, welfare technology, several knowledge spheres or
epistemic cultures meet [22].

Resistance to technology implementation
Resistance is inherent to organizational life [23, 24], and
the literature on resistance stretches across several disci-
plines [25]. According to a recent review of research on
resistance to healthcare information technologies, resist-
ance is under-researched and multifaceted, and relatively
little attention has been paid in understanding it [26].
Resistance to change has mainly been seen as an effort
to maintain status quo and research has traditionally seen
resistance as a negative force that must be overcome [23],
and as a restraining force “that leads employees away from
supporting changes proposed by managers” [27:784].
Resistance to technology implementation is ‘expected’
and can be seen as the flip side of success factors for
innovation which has been emphasized in research on
technology implementation in the Information Systems
(IS) field (see for instance [26, 28]).
Change processes like the implementation of technology

are met by several types of resistance. Resistance is found
at individual, organizational and institutional levels [6–8],
and these levels are inter-connected. Previous research has
for instance shown that traditional organizational constel-
lations may change as a result of technology implementa-
tion [29, 30]. Increased use of technology may change the
work pattern, the division of labour and the interaction
pattern. Previous research also indicates that the imple-
mentation is complicated by a lack of training and lack of
interest from employees [31, 32].
Within the IS field, research on resistance concentrates

on the negative paradigm, focusing on subordinates’ un-
willingness to implement decisions made by the manage-
ment [33, 34]. Resistance occurs if threats are perceived
from the interaction between the object of resistance and
initial conditions [33]. Resistance creates friction, which
has negative connotations and may complicate the imple-
mentation process. Friction is however also an antecedent
to change [35]. As the implementation process proceeds,
the users are likely to make moderations to the set of
initial conditions or the subject of resistance, based on
their experience with the technology. Hence the nature
of the resistance will change through the implementation
process [33], and resistance is not considered as purely
harmful. A further example is the notion of productive re-
sistance [23]. Productive resistance builds on the notion of
resistance as a way of coproducing change and “refers to
those forms of protest that develop outside of institutional
channels” [23:801].

Nilsen et al. BMC Health Services Research (2016) 16:657 Page 2 of 14

In this study, we investigate how resisters think, how
they understand their own resistance and what resisters
do “rather than seeing resistance as fixed opposition
between irreconcilable adversaries” [23:801]. This re-
sistance behaviour is categorized by Coetsee [36] as ap-
athy, passive resistance, active resistance and aggressive
resistance.

Resistance to technology implementation in healthcare
Resistance to increased use of technology in healthcare
is still considered to be under-researched [26, 29]. Lluch
states in a review article on health information technolo-
gies (HIT) that “more information is needed regarding
organizational change, incentives, liability issues, end-users’
HIT competences and skills, structure and work process is-
sues involved in realizing the benefits from HIT” [31:849].
Furthermore, the healthcare field is not one field, and

healthcare technology consists of a wide range of technol-
ogy. Within the healthcare field, hospitals have often been
the preferred empirical setting (see for example [33, 37,
38]), and physicians are the preferred actors under study
(see for example [18, 37]). The municipal healthcare set-
ting differs from that of a hospital, especially due to the
organizational and structural elements of the municipality
itself. The municipality is more complex and consists of
several organizations, weakly tied and embedded in the
larger municipal organization. Still, the levels and the
various actors and units within the greater municipal
organization are linked through the tasks and the users
of the services. Further, the focus on patients’ interests
in healthcare in general and concerning the increased
use of technology, in particular, has led to focus on the
groups who need to collaborate in order to implement
technology [39].
Based on their studies of the implementation of infor-

mation technology (IT) in hospital settings, Lapointe and
Rivard [33] identified five basic components of resistance:
Resistance behaviours (from passive uncooperative to ag-
gressive), the object of resistance (the content of what is
being resisted), perceived threats (negative consequences
that are expected implications of the change), initial
conditions (such as established distributions of power or
established routines) and finally the subject of resistance
(the entity, individual or group, that adopts resistance be-
haviours). They propose a dynamic explanation for resist-
ance to the implementation of technology. The resistance
behaviours result from the nature of perceived threats on
various points in the implementation process. Depending
on what triggers the resistance behaviours, new threats
and consequently, new resistance behaviour emerges. The
perceived threats and the resistance behaviour can be
found at an individual and group level. In this article,
we recognize the five basic components of resistance
identified by Lapointe and Rivard, and define resistance

descriptively as behaviours (attitudes, acts and omissions)
that obstruct or interfere with the process of co-creation
and organizational change.

The case of Digital Night Surveillance
The innovation project at hand is called “Digital Night
Surveillance”, which is a government funded project where
five municipalities, both rural and urban, work with a net-
work of technology developers to develop and implement
the use of sensors and digital communication in nursing
homes and home care services.
The project entailed service development and technol-

ogy development in a co-creation process [21, 40] within a
triple-helix inspired network [41], consisting of (1) a net-
work of small- and medium-size technology enterprises
(SMEs), (2) municipal health and care services, and (3) a
university research group [42]. The overall aim was to de-
velop and implement the best possible solution to the
challenges of night surveillance, in order to enhance se-
curity and quality of care for the service users within the
municipalities’ limited resources [29, 43]. The co-creation
and implementation process was facilitated by a profes-
sional manager or “orchestrator” [42].
The technology to be implemented included sensors

on doors and in electronic security blankets (on mat-
tresses) used during the night. A web-based portal facili-
tated communication via traditional PCs as well as mobile
devices, such as tablet computers and smartphones. Most
of the municipal services already had some welfare tech-
nology installed, such as alarm systems. The novelty of the
new system was tied to the web-based portal into which
different technological applications could be connected
and administered. In this way, technology in different cat-
egories and from different producers could function to-
gether and be programmed and adjusted to the individual
patients’ needs. Alterations could be made based on for
instance variations in needs during the day or due to
the progression of a disease. An alarm went off when
an incident happened. The system was programmed to
send alarm messages to dedicated personnel, and they
received the alarm on either a smartphone, pad or PC,
or a combination of these. They ‘signed out’ the alarm
as they checked on the patient.
The implementation project involved a large number

of stakeholders, and the study of resistance involved ex-
ploring some of these. Data in this study comes mainly
from the healthcare providers on the night shift, managers
on various levels in the municipalities and healthcare insti-
tutions, and the technology developers, who also installed
the equipment and trained the healthcare providers.
Furthermore, the following stakeholders were involved
and/or affected by the project: IT service staff, patients
and families.

Nilsen et al. BMC Health Services Research (2016) 16:657 Page 3 of 14

The home care services and the nursing homes in-
cluded in the project had primary users in need of night
supervision. The residents of the nursing homes suffered
from dementia, and tended to get up at night and wan-
der around, which has been described as one of the most
challenging behaviours to manage [44]. Night surveil-
lance in one form or another (face-to-face or technology
based) was necessary to detect “night wanderers” and
guide them back to bed in order to avoid confusion and
anxiety, avoid the risk of falling and injuries, and protect
other residents from being disturbed and frightened at
night. In the Digital Night Surveillance project, sensors in
blankets and on doors detected and sent a signal if the pa-
tient left the room. The patients did not actively use the
technology; rather the users were the healthcare providers.
The participating municipalities identified a need for

innovation in order to ensure safety at night for the ser-
vice users. Then entered into a contract with a network
orchestrator, a network of technological SMEs and a sci-
ence centre for health and technology in a university, in
order to run an implementation project, which included
both municipal home care services and nursing homes.
The initiative came from the empirical field itself.

Methods
Aim and study design
The aim of this study was to explore resistance to imple-
mentation of welfare technology in five municipalities in

Norway. The design was explorative and draws on a lon-
gitudinal single-embedded case study [45] with elements
of action research. The study was carried out during 2013
and 2014.
A case study is suitable for an explorative, in-depth

study of contemporary events in its real-life context
[45]. The case was a project, organized with sub-projects
in each of the municipalities, with a local project manager
on site. The research took a multi-stakeholder perspective
as both the technology developers in the business net-
work, who also install the technology and train the health-
care providers, and the healthcare providers, on various
levels of the homecare services and nursing homes, were
included in the study. The healthcare providers are the ac-
tual users of the technology and are defined as the users
in our study. The study does not include data from the
end-users.
Three main action research elements were applied: 1)

researcher participation in the project design and planning
activities, 2) researcher participation in (and by occasion
also facilitation of) knowledge sharing and reflection
processes during workshops and meetings, including
presentation of preliminary research findings, and 3) using
focus group interviews not only to collect data but also to
stimulate critical reflection on the co-creation and imple-
mentation process [46, 47].
Table 1 gives an overview of the longitudinal design,

the timeline, the technology, the users and the data col-
lection methods.

Table 1 Design and data collection methods – an overview

Stake-holders Technology Research activities

Q3 2013 Q4 2013 Q1 2014 Q2 2014

Municipality 1 Sensor technology
Alarm system
Web-based portal
Installations: 8

EP
WS
PO
FG

WS
PO
II

WS
PO

WS
PO

Municipality 2 Sensor technology
Alarm system
Web-based portal
Installations: 11

EP
WS
PO
FG

WS
PO
II

WS
II
PO

WS
PO

Municipality 3 Sensor technology
Alarm system
Web-based portal
Installations: 9

EP
WS
PO
FG

WS
PO
II

WS
PO

WS
PO

Municipality 4 Sensor technology
Alarm system
Web-based portal
Installations: 4

EP WS
PO

WS
PO

WS
PO

Municipality 5 Sensor technology
Alarm system
Web-based portal
Installations: 2

EP
WS
PO

WS
PO

WS
PO

Suppliers FG
WS

WS FG
WS

WS

Participants in each workshop 24 33 17 32

Abbreviations: EP Entered the project, II Individual interviews; FG Focus group interviews; PO: Participatory observation; WS Workshops

Nilsen et al. BMC Health Services Research (2016) 16:657 Page 4 of 14

Data collected
The main sources of qualitative data were semi-structured
interviews, both individual and focus group interviews,
and observations in workshops and meetings. Altogether,
data were collected through nine individual interviews,
three focus group interviews and observations on site and
in four workshops. In all, about 50 individuals (including
the five researchers) took part in the workshops and meet-
ings. The researchers facilitated some of the workshops in
order to stimulate co-creation and the production of
process data. Twenty-one individuals were interviewed,
both healthcare providers (from all five municipalities)
and technology developers. All interviewed informants
participated in two or more of the workshops. Some of
the participants in the focus groups were also interviewed
in-depth individually. All participants consented to partici-
pation in the research study.
The selection of informants from the municipalities

for the individual interviews was aided by the project
managers. The inclusion criteria were employees work-
ing as either project manager, middle manager or night
healthcare provider. Eight women and one man were
interviewed in the period from September 2013 to
November 2014. Four technology developers, all male,
participated in a focus group interview in January 2014.
The focus group method was in line with the methodology
used in the project itself, which used the workshops as an
arena for orchestrated interaction, collective reflection,
knowledge sharing and innovation of services [42], thereby
the interviews were an arena for co-creation in themselves
[48]. The in-depth interviews followed a semi-structured
interview guide (Additional file 1) [49, 50] and were car-
ried out as conversations. An interview guide was used as
a checklist at the end of the interview to ensure that all
planned topics were included. The first two focus group
interviews with healthcare providers from three of the mu-
nicipalities were performed as part of a workshop ar-
ranged early in the implementation phase, and were
conducted by four of the researchers. The third focus
group interview was conducted by two of the researchers
with central representatives from the network of technol-
ogy companies. The focus group interviews were con-
ducted face-to-face and lasted for about 90 min each. Both
the in-depth interviews and the focus group interview
were digitally recorded and transcribed verbatim.

Data analysis
Data from the interviews and observations were analysed
and interpreted as inspired by Kvale’s description of the
bricolage approach to data analysis [49]. Analysing data
based on bricolage involves the use of various tech-
niques and concepts during the process. We also used
researcher triangulation [51], which meant that the whole
research team with members from various fields such as

organization and innovation studies, sociology, psych-
ology, nursing, healthcare research and ethics, took part in
the analysis and interpretation process. The main reason
for choosing a researcher triangulation approach was
the need for different perspectives to understand the
complexity of the innovation and co-creation process,
involving five different municipalities, including differ-
ent professional roles, service designs, IT systems, and
local decision-making procedures.
As a first step, following the description of analysis by

Kvale and Brinkmann [49], the transcribed texts from
the interviews were systematically read through in a naïve
manner. A reflexive, open-minded and inductive reading
was pursued, as well as grasping the intuitive meaning of
the text as a whole and to interpret the participants’ ex-
perience and descriptions of the implementation of wel-
fare technology. The themes in the analysis arose in an
iterative process between reading and interpreting by sev-
eral researchers, in order to find meaningful units and
then themes according to the research question [49, 52].
Threats to validity were met by cooperating within the

research team in all phases of the research project, which
ensured an open discussion as well as deep knowledge of
the context. The reliability of the study was strengthened
through researcher triangulation and continuous contact
with the network. Threats to reliability have further been
met by describing the research approach in detail.

Results
At the outset, there were few signs of resistance among
the participants. As the process moved on, various forms
of resistant behaviour emerged, from scepticism of the
usefulness and the functionality and safety of the tech-
nology, to both passive and more active uncooperative
attitudes towards the change of initial conditions, such
as established routines, practices and technological infra-
structure. The perceived threats were often communicated
indirectly, and not always easy to identify, but in many
cases, they were associated with technological instability,
feelings of uncertainty and concerns for the quality of care.
Resistance was found in different groups of participants
and on different levels of the municipal organization. Four
categories of resistance with several subcategories were
identified, as laid out in Table 2.
In the following, the findings will be presented in more

detail and exemplified, starting with organizational issues.

Organizational resistance
Resistance to change in established routines
The surveillance technology was primarily introduced
on the night shift, and only the night shift personnel
were trained to use it. Usually, the employees worked ei-
ther only night shifts or only day/evening shifts, and there
was only brief contact between the shifts. The use of the

Nilsen et al. BMC Health Services Research (2016) 16:657 Page 5 of 14

technology appeared to demand a closer cooperation be-
tween the shifts. For instance, there was a need for the
evening shift to prepare the technology while the patients
were still awake. A night shift worker said: “We need to
have good cooperation with them, so that the mattresses
are placed correctly in the evening and that they are
switched on the way they are supposed to.” Another night
shift worker put it this way:

The day shift must make sure that things work, do
things well, so that I can do a good job. I cannot ask
the patients to wake up and get out of bed so that I
can check that everything is OK in bed. That would be
stupid.

The needs for adjusted routines and better communi-
cation and cooperation between day/evening and night
shifts were soon recognized. However, both project man-
agers and healthcare personnel experienced a lack of
interest and support from the responsible middle man-
agers and unit leaders or ward nurses. As one of the pro-
ject managers answered when asked whether the unit
leader had taken an active role in the project: “No, she
has barely participated and does not take the role. And
she feels it is fine that I have that role”.
This lack of managerial interest and omission to make

the necessary adjustments to established routines (which
was beyond the authority of the project leaders) may be
interpreted as a passive form of organizational resistance
to change, which interfered with, and to some degree
obstructed, the process of co-creation and implementation.

Resistance to necessary competence building
The day shift did not receive any training in how to pre-
pare and use the technology, and would hear about the
project only through information in staff meetings. The
need for training of the day shift personnel was soon

recognized by the project leaders and the other partici-
pants, but the responsible unit leaders did not arrange
for such. The lack of interest from the management in
competence building across shifts resulted in a poor un-
derstanding of the project and the technology on the
part of the day shift. One of the personnel working night
shift declared:

I feel that they do not understand any of this. It is a
«night-shift-thing». (…) and I do not think they follow
up, because it is never talked about. So I hoped we
could have a more thorough conversation about this,
not just two minutes in the staff meeting.

Systemic resistance to communication across groups and
professions
In addition to the lack of communication and cooperation
between shifts, a more general issue emerged concerning
communication, knowledge transfer and organizational
learning. Communication channels across organizational
levels, units and groups of professions within the complex
municipal system were scarce. Those involved in the
implementation of the surveillance technology lacked
sufficient information about, for example, potential
risks. Accordingly, this was an issue in workshops and
inter-municipal meetings. However, not everybody in-
volved could attend the workshops, and some groups –
such as the cleaning staff – were not thought of as having
a role in the implementation process. An example of an
unforeseen risk, which proved to be a problem, was that
cleaning personnel – not being sufficiently informed – on
occasions moved electronic plugs and equipment in order
to clean behind desks and in the corners. Breaking the
electrical circuit might have the effect that sensors or
communication devices shut down, and the error had to
be detected before the system could be made functional
again. The lack of communication channels across groups,
levels and professions may represent an organizational re-
sistance that made it difficult to prepare for unexpected
errors that might obstruct or interfere with a successful
implementation and use. During the workshops, it became
clear that the procedures and written instructions had to
include more groups than initially thought of.

Management resistance to participatory processes
Little by little it became clear that neither the steering
group nor the responsible municipal leaders or their
central IT support departments had taken sufficient
measures to ensure that the necessary infrastructure was
in place to serve the participating homecare units and
nursing homes. It appeared that the municipalities’ IT
support departments had not been included in the initial
phase of the project. This was in spite of the …

RESEARCH ARTICLE

Applications of artificial neural networks in

health care organizational decision-making: A

scoping review

Nida ShahidID
1,2*, Tim Rappon1, Whitney Berta1

1 Institute of Health Policy, Management and Evaluation, University of Toronto, Toronto, Canada, 2 Toronto

Health Economics and Technology Assessment (THETA) Collaborative, University Health Network, Toronto,

Canada

* [email protected]

Abstract

Health care organizations are leveraging machine-learning techniques, such as artificial

neural networks (ANN), to improve delivery of care at a reduced cost. Applications of ANN

to diagnosis are well-known; however, ANN are increasingly used to inform health care

management decisions. We provide a seminal review of the applications of ANN to health

care organizational decision-making. We screened 3,397 articles from six databases with

coverage of Health Administration, Computer Science and Business Administration. We

extracted study characteristics, aim, methodology and context (including level of analysis)

from 80 articles meeting inclusion criteria. Articles were published from 1997–2018 and orig-

inated from 24 countries, with a plurality of papers (26 articles) published by authors from

the United States. Types of ANN used included ANN (36 articles), feed-forward networks

(25 articles), or hybrid models (23 articles); reported accuracy varied from 50% to 100%.

The majority of ANN informed decision-making at the micro level (61 articles), between

patients and health care providers. Fewer ANN were deployed for intra-organizational

(meso- level, 29 articles) and system, policy or inter-organizational (macro- level, 10 arti-

cles) decision-making. Our review identifies key characteristics and drivers for market

uptake of ANN for health care organizational decision-making to guide further adoption of

this technique.

Introduction

As health care systems in developed countries transform towards a value based, patient-cen-

tered model of care delivery, we face new complexities relating to improving the structure and

management of health care delivery; for example, improving integration of processes in care

delivery for patient-centered chronic disease management [1]. Artificial intelligence lies at the

nexus of new technologies with the potential to deliver health care that is cost-effective and

appropriate care in real-time, manage effective and efficient communication among multidis-

ciplinary stakeholders, and address non-traditional care settings, the evolving heathcare work-

place and workforce, and the advent of new and disparate health information systems. With

PLOS ONE | https://doi.org/10.1371/journal.pone.0212356 February 19, 2019 1 / 22

a1111111111

a1111111111

a1111111111

a1111111111

a1111111111

OPEN ACCESS

Citation: Shahid N, Rappon T, Berta W (2019)

Applications of artificial neural networks in health

care organizational decision-making: A scoping

review. PLoS ONE 14(2): e0212356. https://doi.

org/10.1371/journal.pone.0212356

Editor: Olalekan Uthman, The University of

Warwick, UNITED KINGDOM

Received: October 4, 2018

Accepted: January 31, 2019

Published: February 19, 2019

Copyright: © 2019 Shahid et al. This is an open
access article distributed under the terms of the

Creative Commons Attribution License, which

permits unrestricted use, distribution, and

reproduction in any medium, provided the original

author and source are credited.

Data Availability Statement: All relevant data are

within the manuscript and its Supporting

Information files.

Funding: The authors received no specific funding

for this work.

Competing interests: The authors have declared

that no competing interests exist.

the rapid uptake of artificial intelligence to make increasingly complex decisions across differ-

ent industries, there are a multitude of solutions capable of addressing these health care man-

agement challenges; however, there is a paucity of guidance on selecting appropriate methods

tailored to the health care industry[2].

Global health care expenditure is expected to reach $8.7 trillion by 2020, driven by aging

populations growing in size and disease complexity, advancements made in medical treat-

ments, rising labour costs and the market expansion of the health care industry. Many health

systems are reported to struggle with updating aging infrastructure and legacy technologies

with already limited capital resources. In an effort toward moving to value-based care, deci-

sion-makers are reported to be strategically shifting the focus to understanding and better

alignment of financial incentives for health care providers in order to bear financial risk; popu-

lation health management including analyses of trends in health, quality and cost; and adop-

tion of innovative delivery models for improved processes and coordination of care.

Health care organizations are required to be increasingly strategic in their management due

to a variety of system interdependences such as emerging environmental demands and com-

peting priorities, that can complicate decision-making process [3]. According to economy the-

ory, most organizations are risk-aversive [4] and decision-makers in health care can face issues

related to culture, technology and risk when making high-risk decisions without the certainty

of high-return [4, 5]. Patient care and operations management requires the interaction of mul-

tiple stakeholders, for example clinicians, front-line/middle managers, senior level executives

to make decisions on a clinical (e.g. diagnosis, treatment and therapy, medication prescription

and administration), and non-clinical (e.g. budget, resource allocation, technology acquisition,

service additions/reductions, strategic planning) [6].

A white paper published by IBM suggests that with increasing capture and digitization of

health care data (e.g. electronic medical records and DNA sequences), health care organizations

are taking advantage of analyzing large sets of routinely collected digital information in order to

improve service and reduce costs [7]. Reported examples include analyzing clinical, financial and

operational data to answer questions related to effectiveness of programs, making predictions

regarding at-risk patients. The global market for health care predictive analytics is projected was

valued at USD 1.48 billion in 2015 and expected to grow at a rate of 29.3% (compound annual

growth rate) by 2025 [8]. Similarly, global revenue of $811 million is expected to increase 40%

(Compound Annual Growth Rate) by 2021 due the artificial intelligence (AI) market for health

care applications. A subfield of AI, machine learning-as-a-service-market (MLaaS), is expected to

reach $5.4 billion by 2022, with the health care sector as a notable key driver [9].

A recent survey of AI applications in health care reported uses in major disease areas such

as cancer or cardiology and artificial neural networks (ANN) as a common machine learning

technique [10]. Applications of ANN in health care include clinical diagnosis, prediction of

cancer, speech recognition, prediction of length of stay [11], image analysis and interpretation

[12] (e.g. automated electrocardiographic (ECG) interpretation used to diagnose myocardial

infarction [13]), and drug development[12]. Non-clinical applications have included improve-

ment of health care organizational management [14], prediction of key indicators such as cost

or facility utilization [15]. ANN has been used as part of decision support models to provide

health care providers and the health care system with cost-effective solutions to time and

resource management [16].

Rationale

Despite its many applications and, more recently, its prominence [17], there is a lack of coher-

ence regarding ANN’s applications and potential to inform decision making at different levels

Applications of ANN in health care organizational decision-making: A scoping review

PLOS ONE | https://doi.org/10.1371/journal.pone.0212356 February 19, 2019 2 / 22

in health care organizations. This review is motivated by a need for a broad understanding the

various applications of ANN in health care and aids researchers interested in bridging the dis-

ciplines of organizational behaviour and computer science. Considering the sheer abundance

in reported use and complexity of the area, it can be challenging to remain abreast of the new

advancements and trends in applications of ANN [18]. Adopters of ANN or researchers new

to the field of AI may find the scope and esoteric terminology of neural computing particularly

challenging [18]. Literature suggests that current reviews on applications of ANN are limited

in scope and generally focus on a specific disease [19] or a particular type of neural network

[20], or they are too broad (i.e. data mining or AI techniques that can include ANN but do not

offer insights specific to ANN) [10]. The overarching goal of this scoping review is to provide a

much-needed comprehensive review of the various applications of ANN in health care organi-

zational decision-making at the micro-, meso-, and macro-levels. The levels pertain to deci-

sions made on the (micro) level of individual patients, or on a (meso) group level (e.g.

departmental or organizational level) where patient preference may be important but not

essential; and on a wider (macro) level by large groups or public organizations related to allo-

cation or utilization of resources where decisions are based on public interest and reflective of

society as a whole [21]. By means of this review, we will identify the nature and extent of rele-

vant literature and describe methodologies and context used.

Overview

According to an overview by Kononenko (2001), as a sub-field of AI, machine learning pro-

vides indispensable tools for intelligent data analysis. Three major branches of machine learn-

ing have emerged since electronic computers came in to use during the 1950s and 1960s:

statistical methods, symbolic learning and neural networks [22]. ANN have been successfully

used to solve highly complex problems within the physical sciences and as of late by scholars

in organizational research as digital tools enabling faster processes of data collection and pro-

cessing [23]. As practical and flexible modelling tools, ANN have an ability to generalize pat-

tern information to new data, tolerate noisy inputs, and produce reliable and reasonable

estimates [23]. ANN belong to a wide class of flexible nonlinear regression and discriminant

models, data reduction models, and nonlinear dynamical systems [24]. ANN are similar to sta-

tistical techniques including generalized linear models, nonparametric regression and discrim-

inant analysis, or cluster analysis [24]. As a statistical model, it’s general composition is one

made of simple, interconnected processing elements that are configured through iterative

exposure to sample data [23]. Its application is particularly valuable under one or more of sev-

eral conditions: when sample data show complex interaction effects or do not meet parametric

assumptions, when the relationship between independent and dependent variables is not

strong, when there is a large unexplained variance in information, or in situations where the

theoretical basis of prediction is poorly understood [23]. ANN architectures are commonly

classified as feed-forward neural networks (e.g. single-layer perceptron, multi-layer percep-

tron, radial basis function networks) or feed-back, or otherwise referred to as recurrent neural

networks (e.g. Competitive networks, Kohonen’s self-organizing maps, Hopfield networks)

[25].

Artificial neural networks

Originally developed as mathematical theories of the information-processing activity of bio-

logical nerve cells, the structural elements used to describe an ANN are conceptually analogous

to those used in neuroscience, despite it belonging to a class of statistical procedures [23].

Applications of ANN in health care organizational decision-making: A scoping review

PLOS ONE | https://doi.org/10.1371/journal.pone.0212356 February 19, 2019 3 / 22

Basics

ANN can have single or multiple layers [23], and consist of processing units (nodes or neu-

rons) that are interconnected by a set of adjustable weights that allows signals to travel through

the network in parallel and consecutively[13, 26]. Generally ANN can be divided in to three

layers of neurons: input (receives information), hidden (responsible for extracting patterns,

perform most of internal processing), and output (produces and presents final network out-

puts) [27].

A review by Agatonovic-Kustrin & Beresford (2000) describes neural computation to be

powered from the connection of its neurons and that each neuron has a weighted input, trans-

fer function and a single output. The authors state that the neuron is activated by the weighed

sum of inputs it receives and the activation signal passes through a transfer function to pro-

duce a single output. The transfer functions, the learning rule and the architecture determine

the overall behaviour of the neural network [26].

Architecture

Sharma & Chopra (2013) describe the two most common types of neural networks applied in

management sciences to be the feed-forward and recurrent neural networks (Fig 1) in compar-

ison with feed-forward networks common to medical applications [28, 29]. A feed-forward

network can be single-layered (e.g. Perceptron, ADALINE) or multi-layered (e.g. Multilayer

Perceptron, Radial Basis Function) [27, 30]. Sharma & Chopra (2013) describe information

flow in feed-forward networks to be unidirectional from input layer, through hidden layers to

the output layer, without any feedback. Whereas, a recurrent or feedback network involves

Fig 1. Conceptual model of a feed-forward and recurrent neural network.

https://doi.org/10.1371/journal.pone.0212356.g001

Applications of ANN in health care organizational decision-making: A scoping review

PLOS ONE | https://doi.org/10.1371/journal.pone.0212356 February 19, 2019 4 / 22

dynamic information processing having at least one feedback loop, using outputs as feedback

inputs (e.g. Hopfield) [27, 30]. Fig 1 illustrates the two types of networks with three layers

(input, hidden and output).

Learning

In an overview of basic concepts, Agatonovic-Kustrin & Beresford (2000) describe ANN

gather knowledge by detecting patterns and relationships in data and “learn” through experi-

ence. The authors state an artificial neural network learns by optimizing its inner unit connec-

tions in order to minimize errors in the predictions that it makes and to reach a desired level

of accuracy. New information can be inputted into the model once the model has been trained

and tested [26]. Also referred to as the generalized delta rule, backpropagation refers to how an

ANN is trained or ‘learns’ based on data. It uses an iterative process involving six steps: (i) sin-

gle case data is passed to input later, output is passed to the hidden layer and multiplied by the

first set of connection weights; (ii) incoming signals are summed, transformed to output and

passed to second connection weight matrix; (iii) incoming signals are summed, transformed

and network output is produced; (iv) output value is subtracted from known value for that

case, error term is passed backward through network; (v) connection weights are adjusted in

proportion to their error contribution; (vi) modified connection weights saved for next cycle,

next case input set queued for next cycle [23]. Sharma & Chopra (2013) broadly classify train-

ing or ‘learning’ methods in ANN into three types: supervised, unsupervised and reinforced

learning. In supervised learning, every input pattern used to train the network is associated

with an output pattern. The error in computed and desired outputs can be used to improve

model performance. In unsupervised learning, the network learns without knowledge of

desired output and by discovering and adapting to features of the input patterns. In reinforce-

ment learning, the network is provided with feedback on if computation performance without

presenting the desired output [30].

Artificial neural networks and regression models

Neural networks are similar to linear regression models in their nature and use. They are com-

prised of input (independent or predictor variable) and output (dependent or outcome vari-

able) nodes, use connection weights (regression coefficients), bias weight (intercept

parameters) and cross-entropy (maximum likelihood estimation) to learn or train (parameter

estimation) a model [31]. ANN learn to perform tasks by using inductive learning algorithms

requiring massive data sets [18]. A working paper on the use of ANN in decision support sys-

tems states that the structure, quality and quantity of data used is critical for the learning pro-

cess and that the chosen attributes must be complete, relevant, measurable and independent

[18]. The authors further observe that in business applications, external data sources (e.g.

industry and trade databases) are typically used to supplement internal data sources.

Classification and prediction modelling

In the book entitled ‘Data Mining: Concepts and Techniques’, classification is defined as the

process of finding a model that describes and distinguishes data classes or concepts based on

analysis of a set of training data [32]. The authors write that models called classifiers predict

categorical class labels and can be used to predict the class label of objects for which the class
label is unknown. Furthermore, the process is described to consist of a learning step (when a

classification model is constructed) and a classification step (when a model is used to predict

class labels for a given data). Methods include naïve Bayesian classification, support vector
machines, and k-nearest-neighbour classification [32]. Han et al. (2012) suggest that

Applications of ANN in health care organizational decision-making: A scoping review

PLOS ONE | https://doi.org/10.1371/journal.pone.0212356 February 19, 2019 5 / 22

applications can broadly include fraud detection, target marketing, performance prediction,

manufacturing and medical diagnosis.

The available data is divided into two sets for cross-validation: a training set used to develop

a model and a test set, used to evaluate the model’s performance [33, 34]. Appropriate data

splitting is a technique commonly used in machine learning in order to minimize poor gener-

alization (also referred to as over-training or over-fitting) of models [34]. Using more training

data improves the classification model, whereas using more test data contributes to estimating

error accurately [35]. Although a 70:30 ratio can typically be used for training/testing size [36],

various statistical sampling techniques ranging from simple (e.g. simple random sampling,

trial-and-error) to more deterministic (e.g. CADEX, DUPLEX) can be used to split the data

depending on the goals and complexity of the problem [34].

Han and colleagues (2012) write that where classification predicts categorical labels, regres-

sion is used to predict missing or unavailable numerical data values (rather than discrete class
labels). The authors describe regression analysis as a statistical methodology often used for

numeric prediction and encompasses identification of distribution trends based on available

data. An example of numeric prediction is when a model is constructed to predict a continu-

ous-valued function or ordered value (as opposed to a class label). Such a model is called a pre-

dictor model and typically uses regression analysis [32].

ANN can be used to perform nonlinear statistical modeling and provide new alternatives to

logistic regression, the most commonly used method for developing predictive models for

dichotomous outcomes in medicine [31]. Users require less formal statistical training and the

networks are able to detect complex non-linear relationships and interactions between depen-

dent and independent variables. ANN can combine and incorporate literature-based and exper-

imental data to solve problems [26]. Other advantages of ANN, relative to traditional predictive

modeling techniques, include fast and simple operation due to compact representation of

knowledge (e.g., weight and threshold value matrices), the ability to operate with noisy or miss-

ing information and generalize to similar unseen data, the ability to learn inductively from

training data and process non-linear functionality critical to dealing with real-word data [37].

Although ANN do not require knowledge of data source, they require large training sets due

to the numerous estimated weights involved in computation [26]. They may require lengthy

training times and the use of random weight initializations may lead to different solutions [37].

Despite successful applications, ANN remain problematic in that they offer us little or no insight

into the process(es) by which they learn or the totality of the knowledge embedded in them [38].

Several limitations of ANN are identified in the literature: they are limited in their ability to explic-

itly identify possible causal relationships, they are challenging to use in the field, they are prone to

over fitting, model development is empirical potentially requiring several attempts to develop an

acceptable model [37], and there are methodological issues related to model development [31]. In

comparing advantages and disadvantages of using ANN to predict medical outcomes, Tu (1996)

suggests that logistic regression models can be disseminated to a wider audience, whereas ANN

models are less transparent and therefore can be more difficult to communicate and use. Even if

published and made available, the connection weight matrices used in ANN for training a data set

may be large and difficult to interpret for others to make use of, whereas logistic regression coeffi-

cients can be published for any end user to be able to calculate [31].

Methods

The Arksey & O’Malley framework (2005) was adopted to identify the (i) research question,

(ii) relevant studies, (iii) select studies, (iv) chart the data and (v), collate, summarize and pres-

ent findings.

Applications of ANN in health care organizational decision-making: A scoping review

PLOS ONE | https://doi.org/10.1371/journal.pone.0212356 February 19, 2019 6 / 22

Search strategy

Due to the cross-disciplinary nature of our query, the search strategy was designed to identify

literature from multiple databases according to the key disciplines of Health Administration

(Medline and Embase), Computer Science (ACM Digital Library and Advanced Technologies

& Aerospace Database), and Business and Management (ABI/Inform Global and JSTOR). The

selection of the three disciplines reflects the core concepts embedded in our research question:

‘what are the different applications of ANN (Computer Science) in health care organizational

decision-making (Health Administration and Business Management)?’

In consultation with a librarian, a comprehensive search syntax was built on the concepts of

‘artificial neural networks’ applied in ‘health care organizational decision-making’ and tailored

for each database for optimum results. The final search syntax was based on search terms

refined through an iterative process involving examination of a preliminary set of results to

ensure relevance (S1 Appendix). The search strategy was limited to peer-reviewed publications

in English without limitation to the year of publication up until the time of our search (January

2018). Our background search did not identify seminal paper(s) published or advancements

related to our research question, thereby justifying the rationale for not limiting the search to a

specicic start date.

Data collection

Screening of articles occurred in two stages. Identified articles were de-duplicated and

imported to EndNote as a reference manager and to Covidence, a web-based platform, for

screening. The screening inclusion and exclusion criteria were built iteratively via consensus

(NS, TR and WB) (Table 1). Titles and abstracts were first screened to include articles with

keywords related to and/or in explicit reference to artificial neural networks. Articles were

excluded if there was no explicit reference to artificial neural networks; the application was not

in the health care domain or context of health care organizational decision-making, or was not

a publication that was peer-reviewed (e.g. grey literature e.g. conference abstracts and papers,

book reviews, newspaper or magazine articles, teaching courses). Table 1 lists the criteria used

to screen, include or exclude articles in the review.

Subsequently, a full-text review of articles that met the initial screening criteria was con-

ducted on basis of relevance and availability of information for data extraction. In addition to

independent review and extraction of articles, two coders (NS and TR) extracted data from a

subset of articles for consensus, minimization of error, and clarity between reviewers regarding

Table 1. Screening inclusion, exclusion criteria.

Inclusion criteria Exclusion criteria

Titles and

abstracts

Explicit reference to keywords: neural network;

artificial neural network; ANNs;

Does not make explicit reference to artificial

neural networks within the context of healthcare

or medicineMust make reference to ANN if any type of

artificial intelligence or machine learning

techniques used, (e.g. Fuzzy logic, Bayesian

statistics and Self-Organizing Maps, back-

propagation; prediction model; unsupervised

learning)

Publication

Type

Peer-reviewed empirical or theoretical work

(e.g. Journal articles, reports)

Not based on empirical or theoretical work (e.g.

book reviews, newspaper article, course

material); conference papers and abstracts

Setting or

Context

Application in domain of Healthcare and/or

Medicine

Application was not directly related to healthcare

organizational decision making (e.g. speech

recognition)

https://doi.org/10.1371/journal.pone.0212356.t001

Applications of ANN in health care organizational decision-making: A scoping review

PLOS ONE | https://doi.org/10.1371/journal.pone.0212356 February 19, 2019 7 / 22

the choice of data selected for extraction. Information related to study characteristics, aim,

methodology (application, taxonomy, accuracy) and context including organizational level of

analysis (micro-, meso- and macro-) was collected and entered into Microsoft Excel for cate-

gorization and descriptive analysis. Applications of ANN to make decisions directly between

providers and patients was categorized as ‘micro’, any decisions made by a larger group and

not directly related to a patient was categorized as ‘meso’, and decisions beyond an organiza-

tional group (i.e. across different institutions, a system or countries) was categorized as

‘macro’ level of decision-making.

Results

Overall, 3,457 articles were imported for screening, out of which (after removal of duplicates)

3,397 were screened for titles and abstracts to give a total of 306 articles used for full-text

review (Fig 2). Articles were excluded from data collection for reasons such as: there being no

Fig 2. Review process overview. �Articles excluded for the following reasons: Not ANN or suitable synonym (n = 93), use of ANN

unrelated to healthcare organizational decision-making (n = 70), based on iterated exclusion criteria (n = 45), not based on

empirical or theoretical research (n = 9), could not access full-text (n = 9).

https://doi.org/10.1371/journal.pone.0212356.g002

Applications of ANN in health care organizational decision-making: A scoping review

PLOS ONE | https://doi.org/10.1371/journal.pone.0212356 February 19, 2019 8 / 22

explicit reference to ANN being used (91 articles), the application of ANN was not in the con-

text of health care organizational decision-making (68 articles), on basis of study exclusion cri-

teria (53 articles) or …