INCREASING PATIENT PORTAL UTILIZATION

Enhanced Primary Care Electronic Medical Record Education to Improve the Use of Patient
Portals
Brittany Prosev RN, BSN
Mentor: Diana Jacobson PhD, RN, PPCNP-BC, PMHS, FAANP
Arizona State University

1

INCREASING PATIENT PORTAL UTILIZATION

2

Abstract
Background: Health information technology (HIT) refers to the electronic health care systems
organizations used to store, share and analyze healthcare information. A central component of
the HIT infrastructure is an electronic health record (EMR) and although HIT has been shown to
increase enthusiasm for patient care, decrease healthcare costs and improve patient outcomes
overall utilization in the United States (US) remains low.
Methods: At an urban primary care pediatric office located in the southwestern US, an
educational quality improvement project for healthcare practice providers and front office staff
was conducted to increase the utilization of the existing EMR-linked patient portal. The
healthcare providers were asked to complete a pre- and post- survey evaluation of their
knowledge and usage of the patient portal. Provider and patient portal data usage was collected
over a five-month period, September 2019 to January 2020.
Results: Data was analyzed using the Intellectus Statistics software™. Significant results were
found at the conclusion of the project in the number of active patient portal users, web-enabled,
portal logins, labs published/viewed, messages sent, appointment reminders and Santovia
utilization. At the end of the project no significance was found with messages received by the
healthcare providers or staff through the patient portal. Survey results found significant
differences between pre- and post- portal usage. No significance was found on providers’
knowledge on how to web-enable patients. Providers’ also demonstrated no significant change in
their perceptions of the benefit in utilizing the portal in patient care after the educational
intervention. Survey results allowed for additional analysis of commonly utilized portal
functionalities, disease or health topics utilized in Santovia, and suggestions on how to make the
use of the patient portal easier for providers.

INCREASING PATIENT PORTAL UTILIZATION

3

Implications for Health Care Providers: This quality improvement project found that
implementation an EMR-linked patient portal requires a comprehensive practice approach with
structured education sessions. Including all employees can improve patient portal utilization.
This educational project resulted in significant increases in most portal functionalities within 5
months. Further practice change evaluations are needed to evaluate how to improve patient portal
utilization with a larger group of participants in a variety of outpatient settings.
Keywords: Electronic health record, patient portal, advanced provider education, quality
improvement, primary care, pediatrics

INCREASING PATIENT PORTAL UTILIZATION

4

Enhanced Primary Care Electronic Medical Record Education to Improve the Use of Patient
Portals
Healthcare organizations face challenges in delivering high quality, effective and safe
patient care at an affordable cost. Health information technology (HIT) supports the health
information management across computerized systems to promote the secure exchange of
information between consumers, providers, payers and quality monitors (Centers for Disease
Control and Prevention [CDC], 2020). The use of HIT in healthcare, including smartphone
applications have led to an increasing enthusiasm for patient care, decreased healthcare costs and
improved patient outcomes (Fiks et al., 2016; Miller, Latulipe, Melius, Quandt & Arcury, 2016).
An abundance of evidence-based literature and healthcare initiatives have moved organizations
from paper charting to electronic medical records (EMR). An EMR, is a patient-centered digital
version of a patient’s chart, allowing providers to chart and access patient’s medical information
digitally, and often includes a linked patient portal. A patient portal is a web-based system in
which patients can view some of their information contained within their physicians EMR (Fiks
et al., 2016). Although the utilization of EMRs is ubiquitous the utilization of the patient portal
in health organizations and practices remains low (National Coordination for Health Information
Technology [ONC], 2020). In primary care pediatrics, a patient portal, when implemented
correctly can positively influence patient and provider outcomes and improve practices workflow
(Bush et al., 2017; Fiks et al., 2016; Hoogenbosch et al., 2018).
Background and Significance
The use of HIT has become a norm worldwide in clinical interactions. With large
amounts of patient information healthcare organizations are required to maintain, data analytics
have taken a greater role in the day-to-day operations and management of patient information

INCREASING PATIENT PORTAL UTILIZATION

5

(ONC, 2019-a). A central component of the HIT infrastructure is the EMR, or digital version of a
patient’s chart. The digital chart can be shared among multiple different healthcare providers,
agencies, specialists and additional added authorized users (Bush et al., 2017). Main components
of an EMR includes medical history, diagnosis, medications, treatment plans, immunization
schedules, allergies and laboratory and test results. The EMR has evolved even further to often
include an accompanying patient portal. Technology and software developments have improved
creating a consumer-friendly design that permits patients and their families the ability to access
their EMR through an online patient portal (ONC, 2019-a). This portal allows patients to access
their EMR as well as additional functionalities if enabled by the healthcare organization.
Functionality of the patient portal can vary between healthcare institutions, but in most portals,
patients can access their health information, communicate securely and confidentially with
providers, review procedure or bloodwork results, access treatment records and, if applicable,
access evidenced-based literature on their diagnosis (Hoogenbosch et al., 2018). Additional
portal functionalities allowed by some organizations include paying bills, scheduling
appointments and updating individual’s personal healthcare information. The ubiquitous nature
of smart phones and other digital devices have made patient portals even more accessible,
making it easier for patients and their families to access their own digital heath information
(Dalrymple, Rogers, Zach & Luberti, 2016).
The use of technology has increased significantly due to the financial incentives from the
Health Information Technology for Economic and Clinical Care Act in 2009 (Slight et al., 2015).
The Clinical Care Act of 2009, with the aim of improving the processes of HIT in the United
States (US), offered incentives for the adoption and utilization of technological systems like the
EMR. The ONC (2019-a) completed a survey called the Health Information Trends Survey

INCREASING PATIENT PORTAL UTILIZATION

6

(HINTS), to access individuals’ viewing and use of online medical records and the related
smartphone applications in 2017 and 2018. In 2018, the survey found that three out of ten
individuals offered access to their EMR, through the patient portal, viewed their record at least
once within the last year. Even though the viewing of patient’s medical records remains low,
80% of all practices and 95% of all hospitals have adopted certified EMRs in the US (ONC,
2019-a).
Although the use of technology by the US has recently slowly started to increase,
adoption of the patient portal in primary care setting has been slow, and the overall rates of portal
sustained utilization remain low (Fiks et al, 2016). There are several barriers identified in an
organizations adoption of the portal including marketing problems, limited staff commitment and
higher increase of portal adoption in adults than in pediatrics (Fiks et al., 2016). Health
information technology in pediatrics consists of additional laws and regulations governing who
has access to the patients EMR- linked patient portal. As the US continues to accelerate the
implementation of HIT into healthcare, more research is needed to ensure positive patient care
outcomes with the smallest number of adverse consequences.
Purpose. The purpose of this Doctor of Nursing Practice (DNP) project is to increase the
adoption of the patient portal at a privately owned pediatric office in two urban pediatric
outpatient clinics in the southwestern US. This pediatric primary care practice, with
approximately 20,000 total patients, has less than half of the current patients web-enabled and
even fewer families actively using the portal. Currently few advanced practice providers, medical
assistants and front office staff at these two clinic sites are utilizing the portals benefits in patient
care or know how to integrate key portal’s capabilities in the practice’s workflow.

INCREASING PATIENT PORTAL UTILIZATION

7

PICOT Statement. The following evidence based PICOT statement was developed to
help guide a literature search in order to find current evidence related to the improvement of
advanced practice providers interaction with the portal: In the pediatric primary care clinic, does
offering advanced provider education compared to not providing education sessions improve
patient portal utilization and advanced practice providers’ perceptions of using the patient portal
in patient care?
Critical Appraisal and Evidence Synthesis
The three scholarly databases were searched for the literature review included PubMed,
The Cochrane Library and the Cumulative Index of Nursing and Allied Health Literature
(CINAHL). Key words included patient portal OR electronic medical record AND education
sessions OR implementation. The keywords were specifically included to expand the literature to
search to all articles related to the improvement of patient portal utilization with included
education curriculums or education sessions to advanced practice providers. There were nine
articles directly included in this literature review (Appendix A). Two studies were systematic
reviews including an additional 31 articles (Bush et al., 2017; Patelarou et al., 2017). Studies
included a mix of high and low levels of evidence. Current literature regarding the
implementation of a patient portal includes mostly qualitative studies, a lower level of evidence.
All studies included funding sources, adequate sample sizes and included study sample
characteristics. No studies appraised were evaluated to include potential bias. The obtained
literature from the database searches included studies mostly conducted in the US with additional
studies conducted in Iran, the Netherlands and Canada (Fiks et al., 2016; Kooji, Groen, &
Harten, 2018; Shahmoradi, Darrudi, Ariji & Nejad, 2016). A synthesis table was created to
further critically appraise the included studies in order to inform the development of an education

INCREASING PATIENT PORTAL UTILIZATION

8

session (Appendix B). Participants included three studies looking at primary, specialty and
resident/fellow advanced practice providers (Alkureishi, Lee, Webb & Arora, 2018, Bush et al.,
2017; Patelarou et al., 2017) and three studies included parents or guardians (Bush et al., 2016;
Darlymple et al., 2018; Fiks et al., 2016). The remaining two studies included analysis of key
stakeholders (Kooji et al., 2018) and one study examined patients’ perspectives (Nahm et al.,
2017).
Advanced Practice Provider Perceptions
Advanced practice providers play a crucial role in the overall adoption and utilization of
the patient portal within the practice but also by the patients (Bush et al., 2017). Critical appraisal
of the educational interventions used in the studies was analyzed to develop a patient portal
curriculum and improve patient/provider/stakeholder perceptions in order to increase the
likelihood of successful integration of the patient portal into this practice. Key findings, in prior
research, demonstrated that once education sessions were integrated into the practices, the portal
positively influenced providers practice workflow, patient satisfaction, treatment adherence and
improvement in patient-provider communication (Alkureis, Lee, Webb, & Arora, 2015; Bush et
al., 2017; Miller, Latulipe, Melius, Quandt & Arcyry et al., 2016). One meta-analysis analyzing
positive improvements to workflow, found that 80% of the providers noted improvement in
communication and 62% found improvement in medical refills when utilizing the patient portal
(Bush et al., 2018). Enthusiastic responses were also found related to improved patient care
interactions, patient medication and treatment adherence (Alkureis et al., 2015; Kooju et al.,
2018). These results emphasize the importance of using the results of evidence-based practice
research in the development of education training sessions, including the most commonly used
portal capabilities like messaging and medication refills to engage providers and improve their

INCREASING PATIENT PORTAL UTILIZATION

9

willingness to utilize the portal in patient care. Providers are more likely to change care and
implement key points of the education sessions in their patient care when the implementation is
supported by multiple evidence-based practice studies (Patelarou et al., 2017).
Negative considerations for portal implementation found in a few studies, included
perceptions of increased advanced practice providers workload. (Bush et al., 2017; Kooij et al.,
2018, & Shahmoradi et al., 2017). Three studies found that the integration of a patient portal
increased the workload on the advanced practice providers (Bush et al, 2017; Kooij et al., 2018;
Shahmoradi et al., 2017). These studies emphasized the importance of education sessions with
the providers to include a structured workflow strategy to help decrease the burden on the
providers. Additional recommendations included having information technology (IT) personal
available to help answer questions and provide additional training if necessary. In addition,
training should be provided to front office personal to be able answer family questions and webenable patients (Bush et al., 2017). The integration and usage of the EMR linked patient portal
requires the entire organization to be engaged, willing to learn and united on how the patient
portal will be utilized in patient care.
Education Session Curriculum
The second portion of the literature review included key points to include in an education
meeting. Two studies critically appraised included the importance of education sessions
including current laws and regulations such as enabling a portal proxy and adolescent restrictions
(Bush et al, 2016; Kooij et al, 2018). One study found relevance in including mobile device
education as most parent’s access EMR information from their mobile device (Dalrymple et al.,
2018). A majority of the studies included surveys or questionnaires, including 5-point LikertScale questions, to analyze providers’ perceptions of utilization of the patient portal. These

INCREASING PATIENT PORTAL UTILIZATION

10

results helped with the pre- and post- implementation surveys utilized in this project and
engagement methods for the advanced practice provider during the education session.
Santovia. One major inclusion of the advanced practice provider curriculum was the
education platform called Santovia. Santovia is a physician founded health technology that can
be included in an EMR-linked patient portal to allow providers to send education from the EMR,
to the patient or family for review, once a diagnosis has been recorded (Santovia, 2020). The
portal capability was recently activated by the practice, and no prior structured training was
offered to the providers. This available patient education is chosen by the provider from a wide
variety of literature developed by ViewMedica™, EBSCOHealth™ and The American Academy
of Pediatrics. A vast majority of the studies included the importance of the appropriate
digital/health literacy of the patient portal information and display (Bush et al., 2016; Dalrmple
et al., 2018; Kooji et al., 2018; Nahm et al., 2017; Patelarou et al., 2017). Education sent through
the portal should be appropriate for the audience and also relevant to their clinical interaction and
diagnosis. Advanced practice providers should be included in the development of the education
structured for patients and have a choice in what information they choose to send to
patients/parents (Dalrmple et al., 2018; Kooji et al., 2018; Nahm et al., 2017; Patelarou et al.,
2017).
Theoretical & Implementation Framework
The process to successfully implement change has developed across multiple disciplines
with numerous implementation frameworks, models and theories to support the diverse array of
innovative innovations (Mullin et al., 2015). The Model for Evidence-Based Practice Change
(Rosswurm and Larrabee, 1999) was selected to guide this evidence-based project
implementation process (Appendix C). This model is grounded on research literature related to

INCREASING PATIENT PORTAL UTILIZATION

11

evidence-based practice, research utilization, standard language and change theory to guide
practitioners through developing and integrating an evidence-based practice change (Rosswurm
& Larrabee, 1999). This practice model is widely used in the nursing profession and was the
guiding model for this project. The use of this model helped ensure each step of the evidencebased practice change was implemented correctly and smoothly to not only integrate but sustain
the change within the practice.
Social and behavioral theories are widely used for development, implementation and
evaluation of public health and health promotion interventions. The Social Cognitive Theory,
best articulated by Bandura (2007) explains that human behavior has a three-way, dynamic,
interaction that evaluates how personal factors, environmental influences and behavior
continually interact (Appendix D). Each part of the three-way dynamic interaction was utilized in
the development, implementation and evaluation of the educational session and resultant
adoption of the patient portal in patient care. During implementation, it is important to include
teaching all office staff, medical assistants and billing personal while keeping in mind that
previous experiences, and education may influence acceptance outcomes. The staff need to
believe in their own abilities to navigate the EMR. The intervention must provide each learner
with the opportunity to experience successful portal interactions. The environmental learning
conditions must be conducive for behavior change as this is a necessary component of the Social
Cognitive Theory.
Applying Evidence to Practice: Methods and Results
Ethics. There were no foreseeable physical, psychological, social, legal or economic
risks related to this DNP project. Permission to initiate the program was received by Arizona

INCREASING PATIENT PORTAL UTILIZATION

12

State University Institutional Review Board (IRB) in September 2019. Practice owners and lead
IT personal were all consulted and ongoing collaboration was conducted.
Setting. The setting consists of two privately owned pediatric primary care offices
located in an urban southwestern state. Due to a lack of other pediatric practices in the
surrounding areas, each office is busy year around, providing a wide range of services to children
from birth to eighteen years of age. The practice delivers a full spectrum of pediatric healthcare
visits including well, sick, and urgent care hours. There are 17 advanced practice providers and
30 front and back office staff that were invited to the scheduled education sessions. Of the
participants invited, 14 of the advanced practice providers attended the educational session and
most of the 30 front and back office staff attended the second education session.
The practice has an established EMR, eClinicalWorks™, with a connected patient portal.
The eClinicalWorks patient portal is designed to provide patients access to their medical
information over a secure internet portal. This portal when downloaded by family to their
desktop or in a smart phone application is called Healow. The practice can track Healow data
from eClinicalWorks on all portal functionalities.
Program Intervention. This DNP project’s practice change intervention was to provide
two different education sessions, one for the front office staff and one for the advanced practice
providers. In addition, the DNP project included ongoing data collection on the portal’s usage
over time. Data collection was conducted over a 5-month period, starting in September 2019 and
ending in January 2020. The educational session development utilized the results of the literature
review/synthesis after collaboration with the practice owners and the practices lead informational
technology (IT) personal. Data collection, analysis and dissemination of the results were the final
steps on the timeline.

INCREASING PATIENT PORTAL UTILIZATION

13

Following baseline data collection, the first education session conducted included the
advanced practice providers. This session was approximately 1-hour education session in length
and began with the completion of the pre- survey. During the session providers were actively
engaged in a step by step viewing of all capabilities of the practice’s patient portal. The second
education meeting was developed for front office staff, who include receptionists involved in
getting families web-enabled and who answer family’s questions regarding the patient portal. An
educational pamphlet about the patient portal was created for this project and placed in the front
office for staff to distribute to families (Appendix E). A similar education session was provided
to the office staff as to the advanced practice providers. No data or surveys were collected from
the office staff, as data collected from the patient portal (Healow Analytics) was analyzed to
examine the number of patient’s web-enabled and the portals’ utilization.
Instruments and Data Collection. Date from the advanced practice providers was
collected at baseline during the first education session in September, 2019 and again following
the 5-month collection period in January, 2020. The measures analyzed include the advanced
practice provider perceptions and the patient portal utilization. Advanced practice providers’
perceptions will be assessed by a 7-question pre-implementation survey and an 8-question postimplementation survey. Both surveys included a section for providers to leave
comments/suggestions for how to improve the utilization of the patient portal at the practice.
Both the pre- and post-surveys were evaluated for content validity by the other advanced practice
providers not included in the project. Additional data was collected from Healow Analytics
including the number of portal active users, the number of patients web-enabled, total number of
patient logins, laboratory results published/viewed, messages sent/received, appointment
requests and Santovia use.

INCREASING PATIENT PORTAL UTILIZATION

14

Budget. A very minimal budget was required for project implementation. The practice
encouraged the principal investigator to use the previously scheduled meetings and conference
rooms at the practice for the education intervention sessions. The practice printed any required
portal education materials. The main lead IT employee was available once a month for meetings
and to relay any additional question from the advanced practice providers or office staff
(Appendix F).
Results
Data was analyzed using the Intellectus Statistics software™. There was a total of 14
advanced practice providers who completed both pre- and post- surveys. The pre- and postsurveys were analyzed using the Chi-square test of independence and the Man-Whitney U test.
Questions on the surveys and project results are available in Appendix G. Ongoing data analysis
collected from September 2019 to January 2020 from the Healow Patient Portal was analyzed
using the Two Proportions Z-test (Appendix H).
Pre- and Post- Implementation Survey
The surveys were collected by the same number of advanced practice providers for both
the pre- and post- analysis. Three questions were the same on both surveys which allowed for
statistical analysis. Additional questions were asked to qualitatively analyze the providers’
perceptions of utilization of the patient portal.
Quantitative Data
A Chi-square Test of Independence was conducted on two questions; “I know how to
web-enable patients to use the patient portal” and “There were benefits to using the patient portal
to enhance patient care.” The results of the Chi-square test related to how to web-enable patients
were not significant (0.025, χ2(1) = 0.85, p = .357), suggesting that advanced practice providers

INCREASING PATIENT PORTAL UTILIZATION

15

did not show improvement in knowing how to web-enable patients. The Chi-square of
Independence was also utilized to see if there was an increase in the providers’ perception of
utilization of the patient portal to enhance patient care which also demonstrated no statistical
significance (0.025, χ2(1) = 0.58, p = .445). This suggests that the advanced practice providers
did not see benefit to using the patient portal to enhance patient care after the education session.
A two-tailed Mann-Whitney two-sample rank-sum test was conducted to examine
whether there were significant differences in portal usage between the pre- and post- survey. The
two-tailed Mann-Whitney two-sample rank-sum test is an alternative to the independent
samples t-test but does not share the same assumptions (Conover & Iman, 1981). The result of
the two-tailed Mann-Whitney U test was significant based on an alpha value of 0.05, (U =
143, z = -2.20, p = .028). The mean rank for group post intervention was 17.71 and the mean
rank for group pre- intervention was 11.29. This suggests that the distribution of portal usage for
group post intervention was significantly different from the distribution of portal usage indicated
by the pre-survey. The median for post (Mdn = 3.00) was significantly larger than the median for
pre (Mdn = 1.50). There was a significant increase in the portal utilization following the
education intervention and 5-month implementation period (Appendix H).
Qualitative Survey Results
Additional questions on the pre- and post- survey were utilized to evaluate aspects of the
portal most utilized and allowed the advanced practice providers to make suggestions for
increasing portal usage within the practice. Survey questions and results are detailed further in
Appendix G.
Diseases/Healthcare Topics. The common disease and heath topics survey question was
asked to evaluate the education providers were sending, utilization Santoiva, to the patient portal.

INCREASING PATIENT PORTAL UTILIZATION

16

Not all providers answered this question and some providers chose multiple options. Common
topics on the pre-implementation survey included sore throat and health checkup/well visits,
followed by one response each for attention deficit hyperactivity disorder (ADHD),
immunizations, cough and the common cold. The results of the post-implementation survey
included a wider variety of responses including fever reduction, and health checkup/well visits
followed by one response each for asthma, sore throat, urinary tract infection, common cold,
dermatitis, constipation, mollescum, ADHD and hordeolum.
Portal Usage Analysis. Providers recorded the current capabilities of the patient portal
they utilized. All providers circled at least one answer, some choosing multiple. Percentages
were calculated related to the number of times the option was chosen divided by the total number
of providers (n=14). On the pre-implementation survey there were a larger number of options
chosen. Of the respondents on the pre-implementation survey the following rate was chosen per
item: sending lab/testing results (11/14, 78%), patient/provider messaging (9/14, 64%), enhanced
parent/patient knowledge (2/14, 14%), growth chart/tracker (1/14, .07%), and refill requests
(2/14, 14%). There was an unanticipated decrease in the number of options chosen on the postimplementation survey. On the post- implementation survey, the following is the response rate
per item: sending lab and testing results (11/14, 78%), enhancing parent/patient knowledge
(5/14, 37%) and patient/provider messaging (6/14, 42%).
Suggestions. Each survey offered a “fill in the blank” option for the advanced practice
providers to give suggestions regarding how to increase the overall utilization of the patient
portal in the practice. The pre-implementation survey suggestions consisted of increasing the
front office employee’s enthusiasm/engagement in utilizing the patient portal. Most providers
reported that they would be unable to web-enable patients and answer parent questions regarding

INCREASING PATIENT PORTAL UTILIZATION

17

the portal during patient interactions. Another suggestion was to continue ongoing provider
education. Additionally, on the pre-implementation survey, the advanced practice providers
chose the following options on how to make utilization of the patient portal easier: EMR prompts
(8/14. 57%), provide instructions for patients/families (7/14, 50%) and provide education to the
medical assistants (4/14, 28%).
The post implementation survey included a larger response on how to improve portal
utilization. Three providers made suggestions regarding continuing short education meetings on
patient portal utilization to help improve the consistency of the portal in patient interactions. One
provider mentioned a step-by-step guide on how to send information through Santovia would be
beneficial. Two additional providers mentioned they themselves do not have enough time to
web-enable patients and front office/medical assistants helping in this task is necessary.
Additional responses included that Santovia was overall a difficult system to navigate, not user
friendly and the increased ability to toggle between siblings on the EMR would be beneficial.
Patient Portal Analytics
A Two Proportions Z-test was conducted to examine whether there was a significant
difference between the proportions of providers/patient using various portal capabilities before
the September 2019 education intervention and following the five-month education intervention
in January 2020. An abundance of information was available regarding patient portal utilization
available on eClinicalWorks. The chosen EMR-linked patient-portal capabilities (Healow
Analytics)/, and month by month data is available in Appendix H. The total number of patients
for the practice in September 2019 consisted of 20,120 and 21,671 in January 2020.
Two Proportions Z-Test. The Two Proportions Z-test was conducted on multiple patient
portal capabilities to analyze if the advanced practice providers and patients utilizing of the

INCREASING PATIENT PORTAL UTILIZATION

18

portal increased from September 2019 to January 2020. Significant results were found related to
active users, web-enabled patients, total number of logins, laboratory results published/viewed,
messages sent, appointment reminders sent and Santovia utilization. All were significant at (p <
0.001). Non-significant result was found related to messages received (p = 0.296). (Appendix H).
Sustainability
Ongoing implementation and maintenance of the patient portal in practice requires IT
personal to stay up to date on current laws/regulations. Health information technology systems
require updating and maintenance. Office personal and advanced practice providers in this
organization have a member of the staff available to answer ongoing questions and update the
EMR-linked patient portal in the future. It was recommended to create a step-by-step guide, on
portal functionalities to reference by practice employees as needed. Patient portal
discussion/debriefing should be added to future staff meetings to continue improvement of
patient portal utilization within the practice.
Discussion
The purpose of this DNP project was to increase the utilization of an EMR-linked patient
portal. Research supports the improvement of patient and provider outcomes when the patient
portal is successfully utilized and integrated into patient care. Synthesis of literature in this
project demonstrates that education sessions within an organization, including advanced practice
providers and front office staff, can positively influence portal utilization. Following the
implementation of two separate education sessions, significant increase was found in the
utilization of the portal within the practice. Ongoing education and further provider
instruction/interaction within practice with the patient portal is necessary. Further studies are

INCREASING PATIENT PORTAL UTILIZATION

19

needed to evaluate how to improve patient portal utilization with a larger group of participants
and a longer period of data collection.
Project Impact
Results of this DNP project generally correlated with the current evidence synthesis. The
pediatric primary care practice showed improvement in many key elements of portal utilization.
The advanced practice providers overall expressed a willingness to integrate the EMR-linked
patient portal functionalities into their patient care interactions. The EMR has been fully
integrated into the practice but ongoing patient portal integration takes consistent utilization in
order for the advanced practice providers and staff to become proficient in its use. The results
indicated that advanced practice providers did not perceive that the patient portal enhanced
patient care. This result could be related to 50% of the providers choosing “agree” and 50%
choosing “strongly agree” on the pre-implementation survey on this question. No providers at the
start of the project’s implementation disagreed that the patient portal could enhance patient care
and, therefore, there was little change in the providers’ perceptions of the use of the patient portal
to improve care. This DNP project demonstrated that the integration of the patient portal in this
busy pediatric practice is feasible. Further strategies are still necessary to continue to increase the
utilization of the portal in this practice.
Barriers and Facilitators
Barriers for this DNP project included the implementation occurring during the practices
busiest season (Fall and Winter). Providers expressed that they were unable to utilize/practice
sending education through Santovia due to time constraints between patients’ appointments.
Additionally, the biggest barrier was the background laws and regulations required in order to
utilize some aspects of the patient portal. Adolescent, portal proxy and foster families laws

INCREASING PATIENT PORTAL UTILIZATION

20

influenced the education sessions and patients/parents’ access to the portal. Understanding and
collaboration with lead IT personal within the practice was extensive to make sure the practice
was safe from breaches in confidentiality and legal liability.
Facilitators for this project were the practice having education sessions pre-scheduled, IT
personal available and an already established EMR within the practice. The practice owners were
engaged and supportive of the integration of the patient portal. Additionally, integration of the
project consisted of a minimal budget and a feasible ongoing sustainability plan.
Recommendations
Recommendations for future studies include strategic preparations for the education
sessions. Splitting the education sessions from the front office staff and the advanced practice
providers kept the meetings small and dedicated to the most proficient methods for each
audience. Scheduling continuous follow up interaction with the practice during the
implementation stage was helpful. These meetings are important to make sure all settings are
functional, and practice employees are supported. Ongoing communication was necessary with
software representatives from eClinicalworks™ and Santovia™ as well as the practices lead IT
personal. The providers recommended step-by-step implementation guides and additional
meetings to keep discussion regarding the patient portals integrated in the practice’s workflow.

INCREASING PATIENT PORTAL UTILIZATION

21

References
Alkureishi , M. A., Lee, W. W., Webb, S., & Arora, V. (2018, January 4). Integrating patientcentered electronic health record communication training into resident onboarding:
Curriculum development and post-implementation survey among housestaff. Journal of
Medical Internet Research, 4(1). http://dx.doi.org/10.2196/mededu.8976
Bandura, A. (2007). Health promotion from the perspective of social cognitive theory. Journal of
Psychology & Health. 13, 623-649. http://dx.doi.org/10.1080/08870449808407422
Bush, R. A., Connelly, C. D., Perez, A., Chan, N., Kuelbs, C., & Chiang, G. J. (2017). Physician
perception of the role of the patient portal in pediatric health. Journal of Ambulatory
Care Management, 40, 238-245. http://dx.doi.org/10.1097/JAC.0000000000000175
Centers for Disease Control and Prevention. (2020). Data and Heath Information Technology.
Retrieved from https://www.cdc.gov/publichealthgateway/program/resources/data.html
Conover, W. J., & Iman, R. L. (1981). Rank transformations as a bridge between parametric and
nonparametric statistics. The American Statistician, 35(3), 124-129.
Dalrymple, P. W., Rogers, M., Zach, L., & Luberti , A. (2016, November 29). Understanding
internet access and use to facilitate patient portal adoption. Health Informatics Journal,
24, 368-378. http://dx.doi.org/10.1177/1460458216675497
Fiks, A. G., DuRivage, N., Mayne, S. L., Finch, S., Ross, M. E., Giacomini, K., ... Grundmeier,
R. W. (2016). Adoption of a portal for the primary care management of pediatric asthma:
a mixed methods implementation study. Journal of Medical Internet Reserach , 18(6).
http://dx.doi.org/10.2196/jmir.5610

INCREASING PATIENT PORTAL UTILIZATION

22

Hoogenbosch , B., Postma, J., Ginkel, J., Tiemessen , N., Delden, J., & Medendorp, H. (2018,
September 17). Use and the users of a patient portal: Cross-sectional study. Journal of
Medical Internet Reserach , 20. http://dx.doi.org/10.2196/jmir.9418
Kooij, L., Groen, W., & Harten , W. H. (2018). Barriers and facilitators affecting patient portal
implementation from an organizational perspective: Qualitative study. Journal of Medical
Internet Research, 20. http://dx.doi.org/10.2196/jmir.8989
Miller, D. P., Latulipe, C., Melius , K. A., Quandt, S. A., & Arcury, T. A. (2016). Primary care
providers views of paitent portals: Interview study of preceived benefits and
consequences. Jounral of Medical Internet Reserach , 18(1).
http://dx.doi.org/10.2196/jmir.4953
Mullin, H. J., Llimous, F., & Bernrimoj, S. (2015). A systematic review of implementation
frameworks of innovations in healthcare and resulting generic implantation framework.
Health Research Policy and Systems, 13. http://dx.doi.org/10.1186/s12961-015-005-z
Nahm, E. S., Diblasi, C., Silver , K., Zhu, S., Sagherian, K., & Kongs, K. (2017). Patientcentered personal health record and portal implementation toolkit for ambulatory clinics:
a feasibility study. Computers Informatics Nursing, 35, 176-185.
http://dx.doi.org/10.1097/CIN.0000000000000318
Patelarou, A. E., Kyriakoulis, K. G., Stamou , A. A., Laliotis, A., Pistollla, D. S., Mataliotakis ,
M., ... Patelarou, E. (2017, February 11). Approaches to teach evidence based practice
among health professionals: An overview of the existing evidence. Advances in Medical
Education and Practice, 8, 455-464. http://dx.doi.org/10.2147/AMEP.S134475
Santovia. (2020). An integrated all-inclusive health engagement platform. Retrieved from
https://santovia.com/physician-practices/

INCREASING PATIENT PORTAL UTILIZATION

23

Shahmoradi, L., Darrudi, A., Arji, G., & Nejad, A. F. (2017). Electronic Health Record
Implementation: A SWOT analysis. Acta Medica Iranica , 55, 642-649. Retrieved from
http://acta.tums.ac.ir/index.php/acta/article/view/5790
Rosswurm, M.A. & Larrabee, J.H. (1999). A model for change to evidence-based practice.
National Library of Medicine and National Institutes of Health. 31(4), 317-322.
http://10.1111/j.1547-5069.19999.tb00510.x
The Office of the National Coordinator for Health Information Technology. (2019-a) Health IT
Dashboard. Retrieved from https://dashboard.healthit.gov/quickstats/quickstats.php
The Office of the National Coordinator for Health Information Technology. (2019-b). Health IT
legislation . Retrieved from https://www.healthit.gov/topic/laws-regulation-andpolicy/health-it-legislation

Running Head: INCREASING PATIENT PORTAL UTILIZATION

24

Appendix A
Table 1
Evaluation Table
Citation 1

Theory/
Conceptual
Framework

Design/ Method

Sample/ Setting

Major Variables
& Definitions

Measurement/
Instrumentation

Data Analysis

Findings/
Results/Conclusion

Level/Quality of
Evidence;
Decision for
practice/
application to
practice

Alkureishi et al,
2018.
Integrating
Patient-Centered
Electronic
Health Record
Communication
Training into
Resident
Onboarding:
Curriculum
Development
and PostImplementation
Survey Among
Housestaff

EvidenceBased
Implantation
Framework

Design:
Implementation
Study/Qualitative
Study

Setting- University
of Chicago

IV- Patient
centered EMR
curriculum.
Interactive lecture
embedded all new
employee training
curriculum.

11-Item Post
training Survey10 5-Point Likert
Scale
Questionnaire &
1 Open-Ended
Question

Wilcoxon
Rank-Sum
Tests
Kruskal Wallis
Tests

Requiring EMR
training skills to
new employees is
effective. Training
should include best
practices, culture of
humanistic
approaches and
patient centered
EMR use.

LOE: VI

Funding: Grants
from University
of Chicago
Academy of
Distinguished

Purpose: Develop
& Deliver EMR
training

Sample- 158
postgraduate
trainees.
Inclusion- EHR
onboarding for all
employees
including interns,
residents, and
fellows.
Included
Interns 54%
Residents/Fellows45%
PC- 20%
Specialty Trainees79%

DV1- Knowledge
of Barriers, best
practices & ability
to implement
DV2-Effective
Training
DV3-Training
Effectiveness by
Primary Care
Providers

DV1- (3.1 vs 3.9
P<0.001)
DV2- Strongly
Agreed or Agreed
Effectiveness 86.7%
DV3- PC most
effective (P=0.03),
felt training should
be required (P=

Strengths: Direct
analysis of EMR
education
session, includes
breakdown of
type of provider
responses,
Emphasis on
patient-centered
care.
Weaknesses: not
easily identified
theory, analysis
included new
employees. Long
curriculum
development
process

Key: APP- advanced practice providers; CD- chronic diseases; CG- control group; DS-databased searched; DV-dependent variable; ED- emergency
department; EMR- Electronic medical record; GRADE- Grades of recommendation, assessment, development and evaluation; IG- intervention
group; IR- Immunization Records; IS- Implementation Study; IV- independent variable; LOE- Level of evidence; MM; mixed methods studyPRISMA- preferred reporting items for systematic reviews and meta-analyses; QOL- quality of life; RCT- randomized control trial; N-number of
studies; n- number of participants; PP-patient portal; PC- primary care; PHI- personal health information; SA-scheduling appointments; SMsecured messaging; SD- sample demographics; USA- United States of America

INCREASING PATIENT PORTAL UTILIZATION
Medical
Educators &
Arnold P Gold
Foundation

Attrition- 100%

25
DV4- Felt the
training would
change patient
care EMR
interactions

0.10) & would
change practice
(P=0.03).

Feasibility: Easy
implantation, cost
effective.

DV4- 70.9%

Bias: None
Country: USA
Citation 2

Theory/
Conceptual
Framework

Design/ Method

Sample/ Setting

Bush et al,
2017.
Physicians
Perception of
the Role of
Patient Portal in
Pediatric Health

Health Belief
Mode

Design: Mixed
Methods
Approach.
Quantitative and
Qualitative.

Setting: Physicians
from a 520-bed
tertiary pediatric
facility.

Purpose: Increase
understating of
physician
perceptions
regarding the value
of EMR in the
pediatric
environment.

Quantitative
Resultsn=21 providers
including 12
pediatricians & 9
specialists

Funding:
Agency for
Healthcare
research and
Quality.
Bias: None
Reported
Country: USA

Qualitative
Resultsn=6 Physicians- 3
specialists, 2
pediatricians, 1
Hospitalist

HUMAN LEVELTips mnemonic to
enhance patientcentered EMR use
Major Variables
/Definitions

IV- EMR linked
patient portal
application
MyChart.
Quantitative:
DV1Communication of
Health Issues
DV2- Use of
Medical Refills
DV3- Easy to
Enroll Pts
DV4- Perceived
Comprehension of
Impact of Portal

Measurement/
Instrumentation

Data Analysis

Findings/
Results

Surveys- 5 -point
Likert Scale
Interviews
Quantitative
Online Survey
and a Qualitative
Telephone
Interview

IBM SPSS
Software: ChiSquare Tests,
Fisher Exact
Tests

Quantitative
DV1- 80% positive
DV2- 62% positive
DV3- 72% were
neutral
DV4- 60%
improved patient
care
52% Improved
patient adherence
DV5- 80% no
negative impact on
salary

Level/Quality of
Evidence;
Decision for
practice/
application to
practice
LOE: V
Strengths: Shows
both qualitative
and quantitative
Analysis. Easy to
follow data
analysis.
Weaknesses:
Low LOE, small
sample size.
Feasibility/
Conclusion: easy
to implement,
important to
analyze provider
perceptions to
patient portal to

Key: APP- advanced practice providers; CD- chronic diseases; CG- control group; DS-databased searched; DV-dependent variable; ED- emergency
department; EMR- Electronic medical record; GRADE- Grades of recommendation, assessment, development and evaluation; IG- intervention
group; IR- Immunization Records; IS- Implementation Study; IV- independent variable; LOE- Level of evidence; MM; mixed methods studyPRISMA- preferred reporting items for systematic reviews and meta-analyses; QOL- quality of life; RCT- randomized control trial; N-number of
studies; n- number of participants; PP-patient portal; PC- primary care; PHI- personal health information; SA-scheduling appointments; SMsecured messaging; SD- sample demographics; USA- United States of America.

INCREASING PATIENT PORTAL UTILIZATION

26

SD:
Female 54%
Male 48%
Pediatrician 57%
Specialty 43%
Under 50- 54%
Over 50-48%

DV5- No Negative
Impact on Salary

DV6- 43% believed
it increased
workload

DV6- Increased
Workload

Qualitative
Analysis:
Emerging themesportal use varies by
setting, portal
recruitment,
improve patient
communication,
enhanced health
data access, teenage
access needs to be
secure.

Citation 3

Theory/
Conceptual
Framework

Design/ Method

Sample/ Setting

Major Variables
& Definitions

Measurement/
Instrumentation

Data Analysis

Findings/
Results

Bush et al,
2016.
Implementation
of the Integrated
Electronic
Patient Portal in
the Pediatric
Population: A
Systematic
Review

Cognitive and
Behavioral
Theoretical
Framework

Design: SR

Publications
Chosen: 11
publications; cross
sectional surveys,
retrospective
analysis,
qualitative studies,
and usability
testing.

IV- SR Patient
Portal Utilization

Data
AbstractionGRADE

PRISMA

Overall indicated
positive attitudes
toward portal
adoption. Emphasis
should be placed on
health literacy
analysis to improve
portal adoption.

Funding:
Agency for

Purpose: Review
existing research to
define a patient
portal and examine
pediatric patient
portal utilization
through Qualitative
or Quantitative
studies that
focused on EMR
and satisfaction,

Sample:
Publications from
1992- 2014;

Findings
DV1- Portal
Utilization
DV2Patient/Parent
Satisfaction
DV3- Reported
Barriers

structure
education
accordingly.

DV1- Low, Most
commonly used; IR,
SM, SA

Level/Quality of
Evidence;
Decision for
practice/
application to
practice
LOE: V
Strengths: large
sample, relevant
to pediatric PP
utilization
Weaknesses:
Most studies
included large
intuitions not
primary care, not
a variety of MA,

Key: APP- advanced practice providers; CD- chronic diseases; CG- control group; DS-databased searched; DV-dependent variable; ED- emergency
department; EMR- Electronic medical record; GRADE- Grades of recommendation, assessment, development and evaluation; IG- intervention
group; IR- Immunization Records; IS- Implementation Study; IV- independent variable; LOE- Level of evidence; MM; mixed methods studyPRISMA- preferred reporting items for systematic reviews and meta-analyses; QOL- quality of life; RCT- randomized control trial; N-number of
studies; n- number of participants; PP-patient portal; PC- primary care; PHI- personal health information; SA-scheduling appointments; SMsecured messaging; SD- sample demographics; USA- United States of America.

INCREASING PATIENT PORTAL UTILIZATION
Healthcare
Research and
Quality.

attitudes on use,
barriers/facilitators,
adherence or
clinical or health
outcomes.

Bias: None
Reported

PubMed, CINAHL
Plus, PsychINFO,
Academic Search
Premier

27
DV2- Positive
Parent Interaction.
Doesn’t replace
telephone calls.
Overall enthusiastic
outcomes.

DV4- Racial &
Socioeconomic
Disparities
DV5- CD

Inclusion:
Peer reviewed,
published, English
language, original
research and data
analysis, and
patient portal in the
0-18 year old age
group

Country: USA

DV3- Difficulty
interpreting data,
medical
terminology,
missing symptom
checker, concerns
about confidentiality

Feasibility: Easy
to include in
education
intervention.

DV4- Individuals of
color/Medicaid/Low
economic
status=Less likely to
use portal

Exclusion:

Citation 4

Theory/
Conceptual
Framework

Design/ Method

Sample/ Setting

Major Variables
& Definitions

Measurement/
Instrumentation

Data Analysis

Dalrymple &
Rogers, 2018.
Understanding
Internet access
and use to
facilitate patient
portal adoption.

Health Belief
Model

Design: Cross
Sectional Study

Setting: Large
Metropolitan area
in eastern USA
including rural,
urban and
suburban practices.

IV- Implantation
of Patient Portal26 Item Survey

26- Item Survey

SPSS

Purpose:
Understand
information
seeking

large amount
qualitative
designed studies

DV1- Internet
Access

DV5- Increased use
with CD
Findings/
Results

Overall indicated
positive attitudes
toward portal
adoption. Emphasis
should be placed on
health literacy

Level/Quality of
Evidence;
Decision for
practice/
application to
practice
LOE: VI
Strengths: strong
analysis of portal
adoption by
patients, findings
valid,

Key: APP- advanced practice providers; CD- chronic diseases; CG- control group; DS-databased searched; DV-dependent variable; ED- emergency
department; EMR- Electronic medical record; GRADE- Grades of recommendation, assessment, development and evaluation; IG- intervention
group; IR- Immunization Records; IS- Implementation Study; IV- independent variable; LOE- Level of evidence; MM; mixed methods studyPRISMA- preferred reporting items for systematic reviews and meta-analyses; QOL- quality of life; RCT- randomized control trial; N-number of
studies; n- number of participants; PP-patient portal; PC- primary care; PHI- personal health information; SA-scheduling appointments; SMsecured messaging; SD- sample demographics; USA- United States of America.

INCREASING PATIENT PORTAL UTILIZATION
Funding: No
financial
support
Bias: None
Reported by
study
Country: USA

preferences and
internet access
habits of patient
portals to ensure
successful uptake.

Sample: 270
Useable Surveys
where Collected,
minimum of 50
from each clinical
practice.
SD:
Age 21-29- 28%
Age 30-39- 32%
Female- 80%
Male- 19.6%
Inclusion: All
respondents greater
than 21 years or
older, read/write
English- caregivers
of children and
adolescents. Data
collected from
August to
December 2012.
Exclusion: Not
Included
Attrition: Not
Included

28
DV2- Health
Information
Seeking
DV3- Health
Literacy of
Participants
DV4- Attitudes
Towards Portal
Adaption

analysis to improve
portal adoption.
DV1- 95 % reported
having internet
access. 57%
computer or laptop.
36.9% Mobile
Device.
DV2- 42% reported
using the internet to
look up health
related information.
11.5 % said they
look up information
daily. 52% the
respondents were
confident
information was
accurate. 61% trust
information that
comes from the HC
system

Weaknesses:
potential bias
foundparticipations
received gift
cared, no
framework
mentioned, study
design difficult to
assess, no
provider analysis
Feasibility: Easy
to implement into
teaching.

DV3- Substantial
portion of the
population had
marginal or
inadequate health
literacy.
DV4- 33% always
use EMR, 28%
would use
EMR/portal, 29.4%

Key: APP- advanced practice providers; CD- chronic diseases; CG- control group; DS-databased searched; DV-dependent variable; ED- emergency
department; EMR- Electronic medical record; GRADE- Grades of recommendation, assessment, development and evaluation; IG- intervention
group; IR- Immunization Records; IS- Implementation Study; IV- independent variable; LOE- Level of evidence; MM; mixed methods studyPRISMA- preferred reporting items for systematic reviews and meta-analyses; QOL- quality of life; RCT- randomized control trial; N-number of
studies; n- number of participants; PP-patient portal; PC- primary care; PHI- personal health information; SA-scheduling appointments; SMsecured messaging; SD- sample demographics; USA- United States of America.

INCREASING PATIENT PORTAL UTILIZATION

29
sometimes would do
administrative tasks
online

Citation 5

Theory/
Conceptual
Framework

Design/ Method

Sample/ Setting

Major Variables
& Definitions

Measurement/
Instrumentation

Data Analysis

Findings/
Results

Fiks et al, 2016.
Adoption of a
portal for the
primary care
management of
pediatric
asthma: a mixed
methods
implementation
study

EvidenceBased
Implantation
Framework

Design: MixedMethods
Implementation
Study

n= 9,133 invited to
use the portal
IG= 237

IV- MyAsthma
Portal

ACT screening
form,
questionnaires,
interviews

Strata Version
13.1- ChiSquare tests, ttests, Fisher
exact tests,
MannWhitney U
tests

Higher treatment
engagement.

Funding:
Agency for
Healthcare
Research and
Quality, Eunice
Kennedy
Shriver National
Institute of
Child and
Health Human
Development
Bias: Dr. Fiks
and Dr.

Purpose: use of a
patient portal for
pediatric primary
care impact on
management, and
barriers for
implantation
success.

SDm-age- 6-12 years
(p=0.02)
mild or
moderate/persistent
asthma (p.04)
prescription
controller
medication
(p<0.01)
private insurance
(p=0.002)
Setting: 20
practices in 11
difference states.
Inclusion: English
speaking, children
aged 6-12, asthma

DV1- adoption of
the portal
DV2Appointments
DV3- medication
dosage change
DV4Environment
change

DV1- adoption of
the portal
DV2Appointments; 28%
had a asthma-related
primary care visit
DV3- medication
dosage change;
14% increase in
medication changes,
CI 95%.
DV4- 16% reported
to make a change
Qualitative Results
DV1- Well
Coordinated
Workflows
DV2- APP included
in education

Level/Quality of
Evidence;
Decision for
practice/
application to
practice
LOE: III
Strengths:
Nationally
represented in
both USA,
Canada,
comprehensive
SD, large sample
population,
multiple practices
Weaknesses:
Potential bias,
short
implementation
time period,
exclusion criteria
not included
Conclusion:
Higher treatment
engagement,
portal adaption is
better suited for

Key: APP- advanced practice providers; CD- chronic diseases; CG- control group; DS-databased searched; DV-dependent variable; ED- emergency
department; EMR- Electronic medical record; GRADE- Grades of recommendation, assessment, development and evaluation; IG- intervention
group; IR- Immunization Records; IS- Implementation Study; IV- independent variable; LOE- Level of evidence; MM; mixed methods studyPRISMA- preferred reporting items for systematic reviews and meta-analyses; QOL- quality of life; RCT- randomized control trial; N-number of
studies; n- number of participants; PP-patient portal; PC- primary care; PHI- personal health information; SA-scheduling appointments; SMsecured messaging; SD- sample demographics; USA- United States of America.

INCREASING PATIENT PORTAL UTILIZATION
Grundmeier
helped create
software, Dr.
Pace invented
PEERS

30

diagnosis, office
visit during the last
12 months

DV3- Ease of Portal
Use
DV4-Concerned
about perceived
access to a computer

Exclusion: Not
included.

Country: USA,
Canada

long-term
adaption.
Implementation
efforts should
include redesign
and enrollment of
symptomatic
children.
Feasibility: More
feasibility to be
implemented by
individual offices
for long-term
use.

Citation 6

Theory/
Conceptual
Framework

Design/ Method

Sample/ Setting

Major Variables
& Definitions

Measurement/
Instrumentation

Data Analysis

Findings/
Results

Kooij, Groen &
Harten, 2018.
Barriers and
facilitators
affecting patient
portal
implementation
from an
organizational
perspective:
qualitative
study.

The
Consolidation
Framework for
Implementation
Research

Design: Qualitative
Study/ Cross
Sectional Study

Setting: Hospitals
in the Netherlands
7 Hospitals
Included
2 University
medical Centers
3 Teaching
Hospitals
2 General
Hospitals

Medical Centers
with a previous
implemented
patient portal.

Semi-structured
Interviews

Transcribed
InterviewsVerbatim

Multiple barriers
and facilitators
discovered.
Importance of
including multiple
stakeholders in
portal implantation
is essential.

Crol And
Wensing
Model

Purpose: Identify
barriers and
facilitators of
various
stakeholders within
hospital
organizations

Sample:
21stakeholders
were interviewed.

Questionnaires
Excel

DV1- Quotes
DV2- Barriers
DV3- Facilitators

Results
analyzed using
Grol and
Wensing
Model

Total 376 quotes, 26
Barriers and 28
facilitators

Level/Quality of
Evidence;
Decision for
practice/
application to
practice
LOE: VI
Strengths: strong
analysis for
including
stakeholders in
education
intervention,
good included
graphs,
Weaknesses:
interviews
included

Key: APP- advanced practice providers; CD- chronic diseases; CG- control group; DS-databased searched; DV-dependent variable; ED- emergency
department; EMR- Electronic medical record; GRADE- Grades of recommendation, assessment, development and evaluation; IG- intervention
group; IR- Immunization Records; IS- Implementation Study; IV- independent variable; LOE- Level of evidence; MM; mixed methods studyPRISMA- preferred reporting items for systematic reviews and meta-analyses; QOL- quality of life; RCT- randomized control trial; N-number of
studies; n- number of participants; PP-patient portal; PC- primary care; PHI- personal health information; SA-scheduling appointments; SMsecured messaging; SD- sample demographics; USA- United States of America.

INCREASING PATIENT PORTAL UTILIZATION
Funding: Dutch
Cancer Society

31

3 medical
professions, 3
managers and 4 IT
employees.

Bias: None
Reported

Analysis of
Common Themes:
DV2- perceived
usefulness, positive
attitude, knowledge,
motivation to
change,

SD:
N=21
Female- 12%
Male 9%
Age 20-29- 3%
30-39-3%
40-49- 7%
50-59-6%
>60%-2%

Country:
Netherlands

Citation 7

Theory/
Conceptual
Framework

Design/ Method

Nahm, 2017.
Patient-centered
person health
record and
portal
implementation
toolkit for
ambulatory
clinics

Robust
Implementation
and
Sustainability
Model

RCT

Social
Cognitive
Theory

Purpose: To
implement a
training program to
increase poral
adoption and
support for patients
during the
implementation
process.

Attrition: 8
hospitals
approached, 7
included in study
Sample/ Setting

Setting: Diabetes
and Endocrinology
Care Center, part
of an inner-city
university hospital.
Sample:
n= 74 patients
SD:
Age

DV3- lack of
resources, financial
difficulties,
guaranteeing
privacy and
security. eHealth
Literacy, negative
attitude of medical
professionals.
Major Variables
& Definitions

Measurement/
Instrumentation

Data Analysis

Findings/
Results

IV- Patient
Centered Portal
Implementation
Toolkit

Surveys
4-Item SelfEfficacy Tool
Adherence to
Medication-5
Item Medical
Outcomes Study
General
Adherence Scale

SPSS- t tests,
X2 tests,
MannWhitney U
Test, Fisher
Exact Test
Linear Mixed
Models

Online patient
friendly step-by-step
online training
programs can
improve PP low
adoption rates.

DV1- PatientProvider
communication
DV2- Knowledge

Only clinically
significant changeDV1- IV improved
communication at 4

prompts, data
saturation,
potential bias in
answers.
Feasibility: easy
to implement into
training, Surveys
should include
APP current
attitudes/beliefs
on PPs.

Level/Quality of
Evidence;
Decision for
practice/
application to
practice
LOE: I
Strengths: strong
statistics analysis,
high LOE,
Weaknesses:
potential biasmoney given for
survey
completion, not

Key: APP- advanced practice providers; CD- chronic diseases; CG- control group; DS-databased searched; DV-dependent variable; ED- emergency
department; EMR- Electronic medical record; GRADE- Grades of recommendation, assessment, development and evaluation; IG- intervention
group; IR- Immunization Records; IS- Implementation Study; IV- independent variable; LOE- Level of evidence; MM; mixed methods studyPRISMA- preferred reporting items for systematic reviews and meta-analyses; QOL- quality of life; RCT- randomized control trial; N-number of
studies; n- number of participants; PP-patient portal; PC- primary care; PHI- personal health information; SA-scheduling appointments; SMsecured messaging; SD- sample demographics; USA- United States of America.

INCREASING PATIENT PORTAL UTILIZATION
Funding:
University of
Maryland
School of
Nursing and
University of
Maryland
Medical Center
Collaboration
Grant

M 21% F 53%
Race
W 27% B 40%
Other 7%

32
DV3- SelfEfficacy

Components of
Primary Care
Index

weeks (t56= 2.48,
P=0.041) and CG
decreased
communication
(t56=, P=0.35)

DV4- Adherence
to treatment plans

Inclusion: 40 years
or older, clinic
patients, received a
diagnosis of at
least one chronic
disease, could
read/write English,
use internet
independently,
email account,
agreed to activate
the portal.

Bias: None
Reported.
Country: USA

pediatric, small
sample size.
Acute care vs.
Primary Care
analysis.
Feasibility: Not
all companies
will use the
toolkit, easy to
implement
education
regarding patientprovider
communication

Exclusion: Not
Included
Attrition: CG31.6% at 4 weeks,
and 21.1% at 12
weeks.
IVG- 11.1% at 4
weeks and 5.6% at
12 weeks
Citation 8

Theory/
Conceptual
Framework

Design/ Method

Sample/ Setting

Major Variables
& Definitions

Measurement/
Instrumentation

Data Analysis

Findings/
Results

Level/Quality of
Evidence;
Decision for
practice/
application to
practice

Key: APP- advanced practice providers; CD- chronic diseases; CG- control group; DS-databased searched; DV-dependent variable; ED- emergency
department; EMR- Electronic medical record; GRADE- Grades of recommendation, assessment, development and evaluation; IG- intervention
group; IR- Immunization Records; IS- Implementation Study; IV- independent variable; LOE- Level of evidence; MM; mixed methods studyPRISMA- preferred reporting items for systematic reviews and meta-analyses; QOL- quality of life; RCT- randomized control trial; N-number of
studies; n- number of participants; PP-patient portal; PC- primary care; PHI- personal health information; SA-scheduling appointments; SMsecured messaging; SD- sample demographics; USA- United States of America.

INCREASING PATIENT PORTAL UTILIZATION
Patelarou et al,
2017.
Approaches to
teach evidencebased practice
among health
professionals:
an overview of
the existing
evidence.
Funding: Not
Included
Bias: None
Reported

EvidenceBased
Implantation
Framework

SR
Purpose: Analyze
current research to
present approaches
for teaching EBP
among health care
professionals
including MD and
RNs.

N=20 Studies
Database- Medline,
EMBASE
InclusionPublished last 10
years, Included
education
intervention,
Pre/Post
Intervention Style,
Quantitative
estimation of the
effectiveness of the
intervention

Country:
Sample- 6- 609
Health care
professionals,
nurses (n=7),
physicians (n=6),
mixed disciplines
(n=5),
chiropractors
(n=1), Chinese
medicine
institution (n=5)

33
SI IncludingWorkshop,
Lecture, or online
learning
Interventions
Included
I- Interactive
Lectures/ Group
Breakout Sessions
2- 2 Hour Session
3- 20-hour Course
4- Five-hour
educational
workshop with
primary care
doctors

Clinical
Effectiveness &
EBP
Questionnaire
Fresno Test
CAMS Test
EBASE
Questionnaire
Berlin
Questionnaire

MOOSE

EBP & associated
skills in clinical
practice is essential
for patient safety,
QOC, and careers
satisfaction

LOE: II

Online programs
seem to be effective
and comprehensive

Weaknesses:
small sample
size, study
sometimes
difficult to
follow.

1- EBP Knowledge
Improved P<0.001
2- Increased EBP
application, attitudes
and EBP skill level
P<0.01
3- Research
Attitudes P<0.02,
Critical Appraisal
Attitudes P<0.04,
Self- Appraised
Skills P<0.01

Strengths: large
analysis of EBP
teaching methods
to providers,

Feasibility:
Straightforward
curriculum
techniques for
healthcare
providers.

4- Utilization EBM
Resources P=0.001
and EMB
knowledge P=0.000

Setting- Included
Europe, USA,
Taiwan, Canada,
Peru, Iran, Pakistan
& Israel

Key: APP- advanced practice providers; CD- chronic diseases; CG- control group; DS-databased searched; DV-dependent variable; ED- emergency
department; EMR- Electronic medical record; GRADE- Grades of recommendation, assessment, development and evaluation; IG- intervention
group; IR- Immunization Records; IS- Implementation Study; IV- independent variable; LOE- Level of evidence; MM; mixed methods studyPRISMA- preferred reporting items for systematic reviews and meta-analyses; QOL- quality of life; RCT- randomized control trial; N-number of
studies; n- number of participants; PP-patient portal; PC- primary care; PHI- personal health information; SA-scheduling appointments; SMsecured messaging; SD- sample demographics; USA- United States of America.

INCREASING PATIENT PORTAL UTILIZATION

34

Attrition- LS
included 973
records, 20
included
Citation 9

Theory/
Conceptual
Framework

Design/ Method

Sample/ Setting

Major Variables
& Definitions

Measurement/
Instrumentation

Data Analysis

Findings/
Results

Shahmoradi,
Darrudi, Ariji &
Nejad, 2016.
Electronic
health record
implementation:
A SWOT
analysis.

Social Presence
Theory

Design:
Descriptive,
Analytical Study

Setting: 90
Member workforce
from Hospitals
Affiliated to
TUMS. Included
15 hospitals

IVImplementation of
an EMR

Self-Structured
Questionnaire
5-Point Likert
Scale

SPSS
Pearson
Correlation
Coefficient

Some essential parts
or EMR
implementation
include
organizations,
technical, and
resource elements.

Funding:
Supported by
Tehran
University of
Medical
Sciences
Bias: None
Reported
Country: Iran

Purpose: Analysis
of strengths,
weaknesses,
opportunities and
threats in EMR
implementation

DV1- Strengths
DV2- Weaknesses

Sample: 90
managers health
information
management staff
employees
Age- 31-41
f- 67%
m- 33%
B.S Degree- 75%
Employment less
than 15 years- 46%
Inclusion: 90
mangers and health
information
management staffs.
Census and

DV3- Opportunity
DV4- Threat

DV1- Timely and
quick access to
information. Low
volume storage of
information.
Accurate record of
provided services.
Prevention of
medical errors
DV2- lack of
hardware and
infrastructures.
Time-consuming
and difficult to link
the information.

Level/Quality of
Evidence;
Decision for
practice/
application to
practice
LOE: VI
Strengths: unique
insights, clear
breakdown of
EMR strengths,
weaknesses and
threats.
Weaknesses:
Data collection
tools lack validity
and reliability.
Feasibility: Easy
to replicate. No
cost involved.
Legal
implications
included.

Key: APP- advanced practice providers; CD- chronic diseases; CG- control group; DS-databased searched; DV-dependent variable; ED- emergency
department; EMR- Electronic medical record; GRADE- Grades of recommendation, assessment, development and evaluation; IG- intervention
group; IR- Immunization Records; IS- Implementation Study; IV- independent variable; LOE- Level of evidence; MM; mixed methods studyPRISMA- preferred reporting items for systematic reviews and meta-analyses; QOL- quality of life; RCT- randomized control trial; N-number of
studies; n- number of participants; PP-patient portal; PC- primary care; PHI- personal health information; SA-scheduling appointments; SMsecured messaging; SD- sample demographics; USA- United States of America.

INCREASING PATIENT PORTAL UTILIZATION

35

convenience
sampling.

Jeopardizing job
positions.

Exclusion: Not
Included

DV3- Sharing
information. Access
to health
information.
Semantic
coordination and
communication
between internal and
external parts

Attrition: Not
Included

DV4- Lack of
human resources.
Lack of Strategic
planning. Physician/
Clinical Staff
resistance to use
EMR.

Key: APP- advanced practice providers; CD- chronic diseases; CG- control group; DS-databased searched; DV-dependent variable; ED- emergency
department; EMR- Electronic medical record; GRADE- Grades of recommendation, assessment, development and evaluation; IG- intervention
group; IR- Immunization Records; IS- Implementation Study; IV- independent variable; LOE- Level of evidence; MM; mixed methods studyPRISMA- preferred reporting items for systematic reviews and meta-analyses; QOL- quality of life; RCT- randomized control trial; N-number of
studies; n- number of participants; PP-patient portal; PC- primary care; PHI- personal health information; SA-scheduling appointments; SMsecured messaging; SD- sample demographics; USA- United States of America.

Running Head: INCREASING PATIENT PORTAL UTILIZATION

36
Appendix B

Table 2
Synthesis Table
Author

Alkureishi

Bush

Bush

Dalrymple

Fiks

Kooij

Nahm

Patelarou

Shahmoradi

Year
Design
Level of Evidence

2015
IS
VI

2017
MM
V

2016
SR
V

2018
CS
VI

2016
MM/IS
II

2018
QS
VI

2017
RCT
I

2017
SR
I

2017
DS
VI

Study
Characteristics
Demographics: Study
Population
Primary Care APP
Specialty APP
Interns
Residents/Fellows
Parents/Guardians
Patients
Stakeholders
Measurement
Survey/Questionnaires
Likert-Scale
Interviews
Curriculum
EMR Interaction/
Navigation
EBP

X

X

X

X
X
X

X

X
X
X

X

X
X
X

X
X

X
X
X

X
X
X

X
X

X
X
Intervention

X
X

X
X

X

X

X

X

X

Key: APP- advanced practice providers; CS- cross-sectional study; DS- descriptive study; EBP- evidence based practice; EMR- electronic medical
record; IS- implementation study; MM- mixed methods study; RCT- randomized control trial; SR- systematic review

INCREASING PATIENT PORTAL UTILIZATION
In-Person Education
Session
Online Training
Perception Analysis
Provider
Stakeholder
Patient Analysis

37

X

X

X
X

X

X

X

X

X

X
X

X
Dependent
Variable

X
X
X

Patient Portal
Analysis
Effective Training
Influenced Practice
Change
Impact on Patients
Improved Patient Care
or Adherence
Communication
Increased APP
Workload
Chronic Care
Management
Attitudes towards
Portal Adoption
Education Structure
Digital/Health Literacy
Structured Workflow
Education Relevance
Mobile Device Access
APP Included in
Materials Selection
Laws & Regulations
Portal Functionality










































X
X

X
X
X







X

X

X

X
X
X

X



X
X
X

X
X

X
X
X

Key: APP- advanced practice providers; CS- cross-sectional study; DS- descriptive study; EBP- evidence based practice; EMR- electronic medical
record; IS- implementation study; MM- mixed methods study; RCT- randomized control trial; SR- systematic review

Running Head: INCREASING PATIENT PORTAL UTILIZATION
Appendix C
Rosswurm and Larrabee Evidence Based Model Diagrams
Figure 1

(Larabeer, 2009)

38

INCREASING PATIENT PORTAL UTILIZATION
Appendix D
Social Cognitive Theory Model
Figure 2

39

Running Head: INCREASING PATIENT PORTAL UTILIZATION

40
Appendix E

Patient Portal Pamphlet

INCREASING PATIENT PORTAL UTILIZATION

41

Running Head: INCREASING PATIENT PORTAL UTILIZATION

42

Appendix F
Budget
Phase
Preparation

Delivery

Analysis

Activities

Direct Cost

Design and print EHR
materials to give
parents and providers
on accessing the portal
Design/Implement and
lead a provider
education meeting
Rent conference room
available to all
providers at the
Pediatric Office.
Provider time for
presentations. (Cost
Variable- Covered by
the practice)
Utility/Office
Supply/Other
Overhead Costs
Review and complete
analysis of results.
IT personal time to
help collect data for
analysis/ Portal
Maintenance and
Upkeep.

**Covered by the practice or completed by student.

Indirect Cost

subtotal

Total

$50

$0**

$50

$0**
Covered by the
practice
$0**

$0**
Covered by the
practice
$0**
$0**
Provider time
covered by the
practice

$0

$50

INCREASING PATIENT PORTAL UTILIZATION

43

Appendix G
Pre/Post Implementation Survey Results
Question

How many times a
week do you use the
patient portal?
Which of the following
diseases processes or
health topics do you
MOST use the patient
portal for?

Pre-Implementation
Survey (n=14)

Post-Implementation
Survey (n=14)

n (%)
Never- 0%
1-2 times- 50%
3-4 times- 35.7%
5 or more times-14.3%

n (%)
Never-0%
1-2 times- 3 (21.4%)
3-4 times- 3 (21.4%)
5 or more times- 8 (57.1%)

Number of Times
Chosen:
ADHD- 1

Asthma- 1

Sore Throat- 4

Health Check-Ups- 2

Immunizations- 1

Customizable Education- 1

Health Check-Ups- 3

Fever Reduction- 2

Cough- 1

Sore Throat- 1

Common Cold- 1

UTI- 1

Results

Two-Tailed MannWhitney U Test- p=
.028
-------

Common Cold- 1
Dermatitis- 1
Constipation- 1
Mollescum- 1
ADHD- 1
If you do use the
patient portal, which
aspects of the portal do
you current use?

Lab/Testing Results- 12
Patient/Provider
Messages- 9
Enhanced
Patient/Provider
Knowledge- 2
Growth Chart/Tracker1

Stye- 1
Sending Lab/Testing
Results- 11
Enhancing Parent/Patient
Knowledge- 5
Patient/Provider
Messages- 6

-------

INCREASING PATIENT PORTAL UTILIZATION
Do you know how to
web-enable patients?

Do you know how to
use Santoiva to send
patient education?

Refill Requests- 2
No- 12 (85.7%)
Yes- 2 (14.3%)

No- 10
Yes- 4

I feel confidence
sending Santovia
education to patients?

------------

I believe there are
benefits to using the
patient portal to
enhanced patient care?

Agree- 7 (50%)
Strongly Agree- 7 (50%)
Disagree- 0
Strongly Disagree- 0

44
No 10 (71.4%)
Yes 4 (28.6%)

Chi- Square of
Independencealpha value of
0.025, χ2(1) =
0.85, p = .357

-----------

--------

Agree- 7
Disagree- 4
Strongly Agree- 3

---------

Agree- 5 (35.7%)
Strongly Agree- 9 (64.3%)
Disagree-0
Strongly Disagree-0

ChI- Square of
Independencealpha value
0.025, χ2(1) =
0.58, p = .445

INCREASING PATIENT PORTAL UTILIZATION

45

Appendix H
Healow Analytics
Samples
Active Users
September
January
Web Enabled
September
January
Total Number of Logins
September
January
Labs Published
September
January
Labs Viewed
September
January
Messages Sent
September
January
Messages Received
September
January
Appointment Reminders
September
January
Santovia
September
January

Responses

Proportion

SD

z

Two Proportions Z-test
Result

5298
6445

0.26
0.3

0.44
0.46

Z= -7.77

P < .001, CI for α= 0.05 [-0.04, -0.03]

12064
13785

0.6
0.64

0.49
0.48

Z=-7.68

P <.001, CI for α= 0.05 [-0.05, -0.03]

961
1604

0.05
0.07

0.21
0.26

Z= -11.27

P < 0.001, CI for α= 0.05 [-0.03, -0.02]

1132
5250

0.06
0.24

0.23
0.43

Z= -55.80

P < 0.001, CI for α= 0.05 [-0.19, -0.18]

67
330

0
0.02

0.06
0.12

Z= -12.88

P <0.001, CI for α= 0.05 [-0.01, -0.01]

53
197

0
0.01

0.05
0.09

Z=-8.74

P <0.001, CI for α= -0.05 [-0.01, -0.01]

77
69

0
0

0.06
0.06

Z=1.11

P= .296, CI for α= 0.05 [-0.00, 0.00]

967
3166

0.05
0.15

0.21
0.35

Z=-34.59

P <0.001, CI for α= 0.05 [-0.10, -0.09]

12
46

0
0

0.02
0.05

Z=-4.28

P < 0.001, CI for α= 0.05 [-0.00,-0.00]