PATIENT PORTAL

1

Patient Portal Utilization Effect on Patient Engagement

Sara M. Herrera
Edson College of Nursing and Health Innovation, Arizona State University

Author Note
Sara M. Herrera is a registered nurse at Banner University Medical Center- Phoenix in the
Cardiac Treatment Center. She has no know conflict of interest to disclose.
Correspondence should be addressed to Sara M. Herrera, Edson College of Nursing and Health
Innovation, Arizona State University, PO Box 873020, Phoenix, AZ 85004-3020. Email:
smsanto1@asu.edu

PATIENT PORTAL

2
Abstract

Purpose: Patient portals are widely available online applications with many health-related tools
that facilitate patient engagement and enhance communication with providers yet are highly
underutilized. The purpose of this evidence-based practice (EBP) project was to explore an
English and Spanish patient portal educational video's impact on patient engagement in a
Federally Qualified Health Center (FQHC). The social cognitive theory underpins the project
because patients' portal use behavior can change if changing their environmental factors in the
clinic with educational videos.
Methods: The Universities Institutional Review Board granted exempt approval to ensure
human subject protection. The participants included bilingual adult patients in an FQHC who
have access to the internet and email addresses who visited the center during the implementation
period. The tablets in the patient rooms displayed the English and Spanish educational video on
step-by-step instructions on accessing, using the patient portal, and the benefits of use. The
information technology technician pulled aggregate data from the analytics component of the
patient portal before and after the four-week implementation period. The data included total
number of clinic patients, number of active portal users, number of monthly logins, and gender.
The project facilitator used descriptive statistics to compare pre-and post-intervention analytics.
Results: Active portal users increased by 0.22% and monthly logins increased by 390 logins.
Only aggregate data was collected so the statistical significance was not calculated.
Conclusion: This EBP project enhances knowledge on patient portal utilization's impact on
patient engagement and may apply to current practice.
Keywords: patient portal, patient engagement, educational video

PATIENT PORTAL

3

Patient Portal Utilization to Improve Patient Engagement
Healthcare providers consistently confront the challenge of motivating patients to engage
in their care. More health information technology tools, such as patient portals, are accessible to
patients as technology advances. Patient portals provide valuable and easy-to-access resources
that facilitate patient participation in their care. Some of these resources include access to
medication lists, access to health education, and the ability to communicate with providers.
Unfortunately, patient portals are a frequently underutilized resource and a missed opportunity to
allow patients to engage in their health fully.
Background and Significance
Problem Statement
Patient engagement in healthcare can be when a patient is willing to participate in their
care, adhere to their plan of care, or display self-management behaviors (Fleming et al., 2017).
Engaged patients are more likely to have healthy habits and practice preventative health
measures (The Office of the National Coordinator for Health Information Technology [ONC],
2015). Unfortunately, many providers find it challenging to engage patients in their care, know
how much information to share with patients, and know which practices best foster engagement.
Facilitating patient engagement is a complicated task that requires many resources, time,
and effort. Nevertheless, the World Health Organization (WHO) (2021) believes that using
digital health technology can empower individuals to engage in health and self-care
interventions. Health information technologies, such as patient portals, are available in most
health care systems across the United States; however, a study estimates that only 15%-30% of
individuals utilize this platform (Lyles et al., 2020).
Purpose and Rationale

PATIENT PORTAL

4

The Office of Disease Prevention and Health Promotion (ODPHP) (n.d.) reports that
health information technology may foster shared decision-making between providers and
patients, provide tools for patients to manage their health, and increase the efficiency of health
care. Also, patient portal usage has demonstrated improved and increased patient and provider
communication (Dendere et al., 2019). This evidence-based practice (EBP) project aims to
evaluate how patient portal utilization influences patient engagement.
Patients in A Federally Qualified Health Center
Federally Qualified Health Centers (FQHC) provide primary and preventative care to
primarily low-income and underserved individuals (Heisler, 2015). FQHCs are located in areas
with scarce health care providers and provide care to underserved populations, including
uninsured, underinsured, and undocumented. In addition, FQHCs serve many patients with
chronic health conditions such as diabetes, hypertension, and coronary artery disease (Heisler,
2015). Managing chronic health conditions is complex and requires individualized care; and
patient involvement in self-care management (Reed et at., 2019). Therefore, it is vital to utilize
all the resources available to engage patients with chronic conditions in their care to improve
their health outcomes.
Patient Portal Utilization
The WHO (2021) and the ODPHP (n.d.) recognize the value of health information
technology to improve health outcomes and have created health initiatives to increase health
information technology utilization worldwide. For example, one objective of Healthy People
2020 is to "increase the proportion of persons who use electronic personal health management
tools" (ODPHP, n.d.). Patient portal utilization has demonstrated the potential in increasing
patient engagement in health care. A noted benefit of using the patient portal is convenient

PATIENT PORTAL

5

access to medication lists, lab results, and other medical records (Nahm et al., 2020; Reed et al.,
2019). In addition, increased medication adherence and obtaining timely medication refills
occurred with patient portal usage (Dendere et al., 2019; Lyles et al., 2015). Another benefit of
patient portal utilization that patients have reported is easy access to a provider via the secure
messaging component of the portal (Lyles et al., 2015).
Studies have found that people with higher health literacy levels tend to use patient
portals more often than those with lower (Hoogenbosch et al., 2018; Irizarry et al., 2017).
However, the ODPHP (n.d.) claims that health information technology can strengthen health
literacy skills and understanding. The ODPHP (n.d.) also claims that if providers use health
information technology effectively, there is a potential to improve the quality of care and
increase patient participation in decision-making. Increasing uptake of patient portal utilization
aligns with national initiatives and is a tool to empower and motivate patients to engage in their
care.
No Patient Portal Utilization
Patients who do not utilize patient portals express various reasons for not doing so.
Studies have found that some patients are satisfied with and prefer face-to-face or over-thephone interactions with their health providers (Irizarry et al., 2017; Lyles et al., 2020; Reed et al.,
2019). Other studies found that patients were apprehensive about using this type of technology
due to fear of their personal information not being secure (Kisekka & Giboney, 2018; Lyles et
al., 2020). Also, it is vital to note that many patients lack internet access or do not have a
smartphone, which is necessary to utilize online patient portals (Irizarry et al., 2017; Reed et al.,
2019).
Patient Engagement

PATIENT PORTAL

6

Increasing patient engagement is the aim of many health care providers. Patient
engagement is essential to improving the quality of care, improving health outcomes, and
reducing health care costs (Irizarry et al., 2017; Fleming et al., 2017). Individuals often report a
desire to engage in their healthcare but feel they have insufficient resources to increase their
engagement and self-care management skills (ONC, 2015). Increasing patient engagement can
help patients develop self-care management skills and encourage them to take ownership of their
care and improve their health (WHO, 2021).
As the prevalence of chronic health conditions increases worldwide, it is vital for patients
to engage in their care to have successful outcomes. Further, improving patient engagement is
complex and requires time and effort from health care providers, patients, and their family
members. The patient portal is an existing tool that has shown the potential to facilitate
communication with providers and patient participation in care. Therefore, healthcare providers
should encourage their patients to utilize this tool to help achieve the goal of patient engagement.
Internal Evidence
Providers in a FQHC in the Southwestern United States voiced concern regarding the
lack of patient engagement and patient portal utilization in their practice. A review of the
analytics of this FHQC patient portal application, Healow, revealed that only approximately 2%
of patients utilize the patient portal. It also revealed that the most active users are in the age
range of 19-34. The least active users, apart from minors, are in the age bracket of 45-65; there is
no data available for those over 65. Over one month, only three patients viewed their results of
the 1,520 published lab results on the patient portal. Additionally, only 0.05% of patients logged
in to the patient portal over one month, and 0.02% of patients have used it to communicate with
their providers. This inquiry has led to the clinically relevant PICOT question, "For patients in a

PATIENT PORTAL

7

Federally Qualified Health Center, how does utilizing the patient portal compared to not utilizing
the patient portal influence patient engagement in healthcare."
Evidence Synthesis
Search Strategy
The electronic databases used were: PubMed, Cumulative Index of Nursing and Allied
Health Literature (CINAHL), and PsycInfo to conduct a literature search. Keywords included:
patient AND patient portal OR mychart OR patient web portal OR web portal AND engagement
OR involvement OR participation OR satisfaction. The keyword for the population portion of the
PICOT question only included the word patient because the keywords FQHC, underserved,
uninsured, or underinsured yielded little to no relevant results. In addition, all the searches were
limited to a publication date between 2016 to 2021 and peer-reviewed articles. To further refine
the search, quotation marks were used for each MESH term for the keyword patient portal.
The initial search of the keywords in PubMed yielded 3,265 results. After the limiters
were applied and the quotation marks were added to the MESH terms for patient portal, the
results decreased to 474. To further refine the search, results were restricted to meta-analysis,
randomized controlled trails, and systemic reviews, decreasing the final yield to 45 results. The
initial search of the keywords in CINAHL yielded 457 results, after the limiters were applied, the
results decreased to 158. The initial search of the keywords in PsycInfo yielded 178 results and
after the limiters were applied, the results decreased to 50.
The titles and abstracts of the articles of the final yields were reviewed for relevance to
the PICOT question. Only articles in English were reviewed. Articles related to health
technology, but not specific to patient portals were excluded. Critical appraisal was performed on

PATIENT PORTAL

8

20 articles and 10 were selected to be evaluated in depth. The10 articles were chosen based on
relevance to PICO questions and best level of evidence.
Critical Appraisal and Synthesis
The pr10 selected articles were evaluated for quality and strength using the Melnyk &
Fineout-Overhold (2019) rapid critical appraisal tools. All of the studies were published between
the years 2017 and 2020, and the majority took place in the United States. The vast majority of
the studies had reported no explicit biases (see Appendix A, Table A3). The most frequently
utilized conceptual frameworks were the health belief model (HBM) and the social cognitive
theory (SCT) (see Appendix A, Table A3). The studies that reported demographics all included
adults with the mean ages between 46 to 82. Four of the studies had a large sample size, and the
remaining had a smaller sample size (see Appendix A, Table A3).
Four studies were qualitative (see Appendix A, Table A1), and the other six were
quantitative studies (see Appendix A, Table A2). The majority of the articles were a lower level
of evidence, levels five and six. However, two studies were level two on the hierarchy of
evidence. Although quantitative studies are a lower level of evidence, they were still included in
the synthesis because they best described the effects of patient portal utilization.
The measurement tools were heterogeneous. They included chart reviews, questionnaires,
interviews, focus groups, and full-text reviews (see Appendix A, Table A3). The most frequently
used measurement tool was chart reviews. The outcomes identified were also heterogeneous but
were all related to the effect of patient portal utilization or the predictors of patient portal
utilization. The most frequent result of patient portal utilization was patient empowerment, and
the most frequent predictor of patient portal utilization was younger age.
Evidence Influence on Project

PATIENT PORTAL

9

Overall, the literature suggests that patient portal utilization has a positive outcome on
patient engagement, communication, satisfaction, empowerment, and vaccine uptake (Hefner et
al., 2019; Hoogenbosch et al., 2018; Lyles et al., 2020; Nahm et al., 2017; Ochoa et al., 2020;
Risling et al., 2018; Stewart et al., 2020; Szilagyi et al., 2020). The patient portal is a tool that
helps providers better engage their patients in their care. The evidence reveals that people that
are insured, younger, have a higher level of health literacy, and are chronically ill are more likely
to use the patient portal. However, no studies demonstrated improved clinical outcomes with
patient portal utilization. The evidence also indicates that minorities, older adults, and patients
with lower digital literacy are less likely to use the patient portal (Hoogenbosch et al., 2018;
Lyles et al., 2020; Nahm et al., 2017; Ochoa et al., 2020; Portz et al., 2019; Risling et al., 2018;
Wedd et al. 2019).
The literature makes it evident that the development of patient portal education should
focus on older, uninsured, with lower health literacy, and minority populations. Many of the
patients from the FQHC meet these characteristics. In addition, effective patient portal education
should include the benefits of portal usage, online modules, in-person assistance and should be in
the patient's preferred language (Lyles et al., 2020; Nahm et al., 2017; Portz et al., 2019). This
search led to a project to increase patient portal usage in a FQHC by using a bilingual
educational video to give patients step-by-step instructions on using the patient portal and
educate them on the benefits of using the patient portal.
Theoretical and Implementation Framework
Theory Application
Theories are used in research to explain a phenomenon, facilitate the analysis and
interpretation of findings, and generalize the findings to similar studies (Fain, 2017). The

PATIENT PORTAL

10

theoretical framework selected for this project was the Social Cognitive Theory (SCT) (see
Appendix B, Figure B1). Bandura's (1986) SCT postulates that human behavior is explained by
personal factors, environmental factors, and behavioral factors. Personal factors include
knowledge, attitudes, and outcome expectations. Environmental factors include perceived social
norms, community, and influence on others. Behavioral factors include skill and self-efficacy
(Bandura, 1991). Self-efficacy is a crucial component in the theory that influences the degree of
effort one will apply and how much one will persist in their actions (Bandura, 1991). All of the
factors mentioned above continuously impact one another and determine behavior.
The SCT applies to the PICOT question. The personal factors would be a patient's
knowledge and attitude towards potential benefits related to the patient portal. Environmental
factors could be perceived barriers and the ability to influence those barriers. Lastly, behavioral
would be a patient's belief that they can utilize the patient portal. These factors would influence
one another and determine the patient's ability to use the patient portal and influence their
engagement in their health.
Implementation Framework
Implementation frameworks guide and execute an evidence-based project. The
Rosswurm and Larrabee Model was selected to direct the implementation of this project (see
Appendix B, Figure B2). This model has been successfully utilized in nursing evidence-based
practice several times. Also, it provides six linear steps to implement a change to practice while
allowing for flexibility if any step needs revisiting. Another added quality is that the first few
steps include stakeholders, identifying a problem, identifying potential interventions, and
synthesizing the best evidence, which aligns with the first steps completed in the project
(Rosswurn & Larrabee, 1999).

PATIENT PORTAL

11

The first step is to assess; in this step, the site champion identifies the lack of patient
portal usage by communicating with stakeholders, the project facilitator collecting internal data,
and comparing it with external data (Rosswurn & Larrabee, 1999). The following step links
where the project facilitator selected the outcome of patient engagement. The next step is to
synthesize the best evidence completed in the critical appraisal component. As the project
facilitator followed the linear steps, the intervention was planned, evaluated, and integrated into
standard practice (Rosswurn & Larrabee, 1999).
Methods
Human Subject Protection
Arizona State University's (ASU) Institutional Review Board (IRB) granted exempt
approval for this EBP project to ensure human subject protection. The patients were consented to
receive care with the clinic's established consent form by the front desk clerical staff while the
patient registered. The minimal potential risk of participating was frustration with using
technology. The information technology technician and the project facilitator only aggregate data
from the analytics component of the patient portal application; the information is stored in the
application and is stored for as long as the clinic continues to use this application. The data
collected had no patient identifying factors, and the names of the active and inactive portal users
were not known.
Population and Setting
The project site was a FQHC located in the Southwestern United States in a predominantly
Hispanic community that serves primarily Hispanic families in the surrounding neighborhoods.
FQHCs provide primary and preventative care to primarily low-income and underserved
individuals (Heisler, 2015). The goal of this FQHC is to provide excellent healthcare regardless of

PATIENT PORTAL

12

financial or insurance coverage status. Providers in this FQHC voiced concern regarding the lack
of patient engagement and patient portal utilization in their practice. A provider in a FQHC,
reported that she has never used the patient portal to communicate with her patients (Provider,
personal communication, February 8, 2021). Instead, they usually call the office and leave a
message with the receptionist or medical assistant (MA). Then, the MA relays the message to the
provider, and either the provider or MA returns the patient’s call. Another provider in a FQHC
reported that she believes many patients are dissatisfied when calling the clinic because the hold
time is usually very long (personal communication, February 8, 2021). She reported an opportunity
to improve patient-provider communication and patient engagement. This clinic employs
providers who are physicians, physician assistants, and nurse practitioners. Other employees
include medical assistants, medical interpreters, receptionists, and an information technology
technician.
The providers at this FQHC were stakeholders because increasing patient portal utilization
can impact how they communicate with their patients. The medical director of this clinic was also
a stakeholder as she is the site champion and the individual who expressed interest in exploring
patient portal utilization. The medical director influenced other providers to encourage their
patients to use the patient portal. Further, the medical assistants and receptionists were instrumental
because they helped educate patients on the benefits of patient portal use. Other stakeholders
included the patients at this FQHC. Their involvement in the project was critical as they viewed
the educational tool and utilized the patient portal.
Expected Impact
Patient portals are a widely available yet highly underutilized health information
technology tool to facilitate engagement. The exhaustive literature review affirms that patient

PATIENT PORTAL

13

portals have many benefits but that many factors impact patients' use. Educating patients on the
benefits of portal usage and giving them instructions on how to use the portal using online videos
in the patient's preferred language can influence their utilization. This EBP project evaluated an
educational video's influence on patient portal utilization in a FQHC. Going forward, the impact
of increased patient portal use may improve patient-provider communication and ultimately
increase patient engagement and satisfaction at this FQHC.
Project Description
After IRB approval, the project facilitator developed the English and Spanish educational
video on step-by-step instructions on accessing and using the patient portal and the benefits of
patient portal use. First, the project facilitator typed out the video scripts, and a native Spanish
speaker uninvolved with the project proofread the Spanish script to assure accuracy. Then
aggregate data from the analytics component of the patient portal application was pulled one month
before the start of the project by the clinic's information technology technician. Then the videos
were uploaded to the tablets located in the patient rooms and played on a loop of videos. The
tablets displayed the video for four weeks. After four weeks, to assess the effectiveness of the
video intervention, the project facilitator pulled aggregate data.
Data Collection
The information technology technician and project facilitator pulled data from the analytics
component of the Healow patient portal application. The principal investigator, project facilitator,
and site champion reviewed the data. The data collected had no patient identifying factors. The
data collected included: the total number of clinic patients, number of active portal users, number
of monthly logins, and gender. The data collected did not include the names of active portal users.

PATIENT PORTAL

14

The plan for data analysis was descriptive statistics for the pre- and post-intervention analytics.
The pre and post-statistics were compared to assess the significance of the intervention.
Budget
The total budget for this EBP project was about one hundred and seven dollars. Appendix
C lists the budget allocation. The project facilitator created the educational video using Adobe
Premiere Rush, which usually costs $10 per month but is available at no cost to ASU students. The
patients viewed the video while waiting to be seen; therefore, there is no estimated loss of
productivity time. Printing handouts cost about $60. The project facilitator received no funding
from an external source for this project.
Results
A month before the intervention, the number of active patient portal users was 1,981 or
3.27%. After the educational video intervention, the number of active portal users was 2,296 or
3.49%. There was a 0.22% increase in active patient portal users. Further, the number of monthly
logins was extracted for the month before, the month during, and the month after the
intervention. There were 1,367 monthly logins, 315 male and 1052 female, the month before the
intervention. The month after the intervention, there were 1,778 monthly logins; 393 males and
1,364 females. The monthly logins increased by 390 logins, 78 males and 312 females. Of note,
this FQHC serves a significant female population though the aggregate data did not provide
specific numbers of total women served. The project facilitator did not gather individual user
data, so the statistical significance of the results was not calculated..
Impact and Sustainability
The providers were receptive to the intervention since the clinic leaders aim to increase
patient portal utilization. The videos also helped providers learn about the impact of patients

PATIENT PORTAL

15

utilizing the patient portal. Unfortunately, the patient portal went through a major update after
the project facilitator created the videos. Therefore, the site did not use the videos after the
intervention period. However, the site champion hopes to have the videos recreated with up-todate information. In the future, another student can update the videos and work on increasing
uptake of the patient portal as a legacy project.
Discussion
This cost-effective, low-risk DNP project highlighted the significant gap in active patient portal
users at this FQHC. The findings show the potential that educational videos may increase uptake
of patient portal utilization, especially in women. There was a more significant increase in
monthly logins by women. However, it is essential to note that the FQHC sees a substantial
population of women; therefore, the finding might be skewed. These findings align with previous
studies that found in-person or video tutorials in patients’ preferred language increased patient
portal utilization (Lyles et al., 2020; Nahm et al., 2017; Portz et al., 2019).
Limitations
A limitation of the project was the patient portal software was updated after the project
facilitator created the educational videos. This update impacted the applicability and
sustainability of the videos. Another limitation was a COVID-19 surge during the intervention,
leading to minimal in-person visits. Further, the data only included aggregate data, so the
statistical significance was not calculated.
Recommendations
Future recommendations include gathering individual patient data to calculate the
statistical significance. Future data can also include qualitative data such as patient satisfaction in
patients who are active patient portal users compared to non-active patient portal users.

PATIENT PORTAL

16
References

Bandura, A. (1986). Social foundation of thought and action: A social cognitive theory. Prentice
Hall.
Bandura, A. (1991). Social cognitive theory of self-regulation. Organizational Behavior and
Human Decision Processes, 50(2), 248-287. https://doi.org/10.1016/07495978(91)90022-L
Dendere, R., Slade, C., Burton-Jones, A., Sullivan, C., Staib, A., & Janda, M. (2019). Patient
portals facilitating engagement with inpatient electronic medical records: A systematic
review. Journal of Medical Internet Research, 21(4), e12779.
https://www.jmir.org/2019/4/e12779/
Fain, J. A. (2017). Reading, understanding, and applying nursing research (5th ed.). F. A. Davis
Company.
Fleming, M. D., Shim, J. K., Yen I. H., Thompson-Lastad, A., Rubin S., Van Natta M., & Burke,
N. J. (2017). Patient engagement at the margins: Health care providers’ assessments of
engagement and the structural determinants of health in the safety-net. Social Science &
Medicine, 183, 11-18.
https://www.sciencedirect.com/science/article/pii/S0277953617302587
Han, H. R., Gleason, K. T., Sun, C. A., Miller, H. N., Kang, S. J., Chow, S., Anderson, R., Nagy,
P., & Bauer, T. (2019). Using patient portals to improve patient outcomes: Systematic
Review. JMIR Human Factors, 6(4), e15038. https://doi.org/10.2196/15038
Health Communication Capacity Collaborative. (n.d.). Social cognitive learning theory [Image].
https://sbccimplementationkits.org/sbcc-in-emergencies/social-cognitive-learning-theory/

PATIENT PORTAL

17

Hefner, J. L., MacEvan, S. R., Blitz, A., & Sieck, C. J. (2019). Patient portal messaging for care
coordination: A qualitative study of perspectives of experienced users with chronic
conditions. BMC Family Practice, 20(1), 57. https://doi.org/ 10.1186/s12875-019-0948-1
Heisler, E. J. (2015). Federal health centers: An overview. Congressional Research Service.
https://congressional-proquestcom.ezproxy1.lib.asu.edu/congressional/result/pqpresultpage.gispdfhitspanel.pdflink/$2fa
pp-bin$2fgis-congresearch$2f1$2ff$2fa$2fe$2fcrs-2015-dsp0149_from_1_to_38.pdf/entitlementkeys=1234%7Capp-gis%7Ccongresearch%7Ccrs2015-dsp-0149
Hoogenbosch, B., Postma, J., De Man-Van, J., Tiemessen, N. A., Van Delden, J., & Van OsMedendorp, H. (2018). Use and users of a patient portal: Cross-sectional study. Journal
of Medical Internet Research, 20(9), e262. https://doi.org/10.2196/jmir.9418
Irizzary, T., Shoemake, J., Nilsen, M. L., Czaja, S., Beach, S., & Dabbs, A. D. (2017). Patient
portals as a tool for health care engagement; A mixed-method study of older adults with
varying levels of health literacy and prior patient portal use. Journal of Medical Internet
Research, 19(3), e99. https://doi.org/10.2196/jmir.7099
Kisekka, V., & Giboney, J. S. (2018). The effectiveness of health care information technologies:
Evaluation of trust, security beliefs, and privacy as determinant of health care outcomes.
Journal of Medical Internet Research, 20(4), e107. https://doi.org/10.2196/jmir.9014
Lyles, C. R., Nelson, E. C., Frampton, S., Dykes P. C., Cemballi, A. G., & Sarkar, U. (2020).
Using electronic health record portals to improve patient engagement: Research priorities
and best practices. Annals of Internal Medicine, 172(11), S123-S129.
https://www.acpjournals.org/doi/10.7326/M19-0876

PATIENT PORTAL

18

Lyles, C. R., Sarkar, U., Schillinger, D., Ralston, J. D., Allen, J. Y., Nguyen, R., & Karter, A. J.
(2015). Refilling medication through an online patient portal: Consistent improvements
in adherence across racial/ethnic groups. Journal of the American Medical Informatics
Association, 23(e1), e28-e33. https://doi.org/10.1093/jamia/ocv126
Melnyk, B. M., & Fineout-Overholt, E. (2019). Evidence-based practice in nursing and
healthcare (4th ed.). Wolters Kluwer.
Nahm, E. S., Diblasi, C., Gonzales, E., Silver, K., Zhu, S., Sagherian, K., & Kongs, K. (2017).
Patient-centered personal health record and portal implementation toolkit for ambulatory
clinics. Computers, Informatics, Nursing, 35(4), 176-185.
https://doi.org/10.1097/CIN.0000000000000318
Nahm, E. S., Zhu, S., Bellantoni, M., Keldsen, L., Charters, K., Russomanno, V., Rietschel, M.,
Son, H., & Smith, L. (2020). Patient portal use among older adults: What is really
happening nationwide? Journal of Applied Gerontology, 39(4), 442-450.
https://doi.org/10.1177/0733464818776125
Ochoa, C., Baron-Lee, J., Popescu, C., & Busl, K. M. (2020). Electronic patient portal utilization
in neurology patients and association with outcomes. Health Informatics Journal, 26(4),
2751-2761. https://doi.org/10.1177/1460458220938533
Office of Disease Prevention and Health Promotion. (n.d.). Health communication and health
information technology. https://www.healthypeople.gov/2020/topicsobjectives/topic/health-communication-and-health-information-technology
Office of the National Coordinator for Health Information Technology. 2015. Federal health IT
strategic plan 2015-2020. https://www.healthit.gov/sites/default/files/9-5federalhealthitstratplanfinal_0.pdf

PATIENT PORTAL

19

Portz, J. D., Bayliss, E. A., Bull, S., Boxer, R. S., Bekelman, D. B., Gleason, K., & Czaja, S.
(2019). Using technology acceptance model to explore user experience, intent to use, and
use behavior of a patient portal among older adults with multiple chronic conditions:
Descriptive qualitative study. Journal of Medical Internet Research, 21(4), e11604.
https://doi.org/10.2196/11604
Reed, M. E., Huang, J., Millman, A., Graetz, I., Hsu, J., Brand, R., Ballard, D. W., & Grant, R.
(2019). Portal use among patients with chronic conditions: Patient-reported care
experiences. Medical Care, 57(10), 809-814.
https://doi.org/10.1097/MLR.0000000000001178
Risling, T., Martinez, J., Young, J., & Thorp-Froslie, N. (2018). Defining empowerment and
supporting engagement using the patient views from the citizen health information portal:
Qualitative study. Journal of Medical Internet Research, 6(3), e43.
https://doi.org/10.2196/medinform.8828
Rosswurm, M. A., & Larrabee, J. H. (1999). A model for change to evidence-based practice
[Image]. Journal of Nursing Scholarship, 31(4), 317-322. https://doi.org/10.1111/j.15475069.1999.tb00510.x
Stewart, M. T., Hogan, T. P., Nicklas, J., Robinson, S. A., Purington, C. M., Miller, C. J.,
Vimalananda, V. G., Connolly, S. L., Wolfe, H. L., Nazi, K. M., Netherton, D., &
Shimada, S. (2020). The promise of patient portals for individuals living with chronic
illness: Qualitative study identifying pathways of patient engagement. Journal of Medical
Internet Research, 22(7), e17744. https://doi.org/10.2196/17744
Szilagyi, P. G., Albertin, C., Casillas, A., Valderrama, R., Duru, K., Ong, M. K., Vangala, S.,
Tseng, C. H., Rand, C. M., Humiston, S. G., Evans, S., Sloyan, M., & Lerner, C. (2020).

PATIENT PORTAL

20

Effect of patient portal reminders sent by a health care system on influenza vaccination
rates: A randomized clinical trial. JAMA Internal Medicine, 180(7), 962.
https://doi.org/10.1001/jamainternmed.2020.1602
Wedd, J., Basu, M., Curtis, L. M., Smith, K., Lo, D. J., Serper, M., Wolf, M. S., Parker, R.,
Patzer, R. E. (2019). Racial, ethnic, and socioeconomic disparities in web-based patient
portal usage among kidney and liver transplant recipients: Cross-sectional study. Journal
of Medical Internet Research, 21(4), e11864. https://doi.org/10.2196/11864
World Health Organization. (2021). Self-care intervention for health.
https://www.who.int/health-topics/self-care#tab=tab_1

PATIENT PORTAL

21
Appendix A
Evaluation and Synthesis Tables

Table A1
Evaluation Table of Quantitative Studies
Citation

Theoretical/
Conceptual
Framework

Design/
Method/
Purpose

Sample/Setting

Variables

Measurement/
Instrumentation

Data
Analysis

Results/
Findings

Nahm et al. (2017)
Patient-centered
personal health
record and portal
implementation
toolkit for
ambulatory clinics

PRISM

Design:
2 Phase
Randomized
Controlled
Trail

n= 74

IV: PP
implementation
toolkit (general
and plus)
CG: General
toolkit only

Face-to-face
education on PP,
brochures, demo
videos, learning
modules

Mann-Whitney
U test

Both providers
and patients
need PP
training

PP knowledge test

Linear mixed
models

Country:
USA
Funding:
University of
Maryland School
of Nursing

SCT

Purpose: To
implement a
PP
implementatio
n toolkit in a
large
underserved
area and
assess the
impact

Demographics:
Men: 21 (28.4%)
Female: 53 (71.6%)
Mean age: 57.4
Setting
Large diabetes and
endocrinology
ambulatory care
center in an
underserved area
Inclusion
40 years or older
Diagnosed with at
least 1 chronic
disease
Can read/write
English
Can use internet
Has an email account

DV1: PP
knowledge
DV2:
communication
DV3: selfefficacy
DV4:
adherence

Components of
Primary Care Index
Self-efficacy scale
Medical Outcomes
Study General
Adherence Scale

Fisher exact
test

Level of
Evidence;
Application to
practice/
Generalization
LOE: II

Strength: Higher
LOE

Older adults
may need
additional PP
training

Weakness: small
sample size

An online
education
module in
addition to
face-to-face
education
improved
patientprovider
communicatio
n

Feasibility:
Learning modules
may be difficult to
imbed into PP it they
are currently nonexistent

Harm: none

Utility to PICOT:
setting is similar to
the population in the
PICOT

Key: CG- control group DS- database searched DV- dependent variable EC- exclusion criteria EHR- electronic health record HBM- health belief model IC- inclusion criteria IVindependent variable LOE- level of evidence n- number of participants N- number of studies NPPU- non patient portal user OP- outpatient PP- patient portal PPU- patient portal
user pt- patient RCT-randomized clinical trial SCT- Social Cognitive Theory tx- transplants UCLA- University of California Lost Angeles UNOS- United Network for Organ
Sharing

PATIENT PORTAL
Citation

Theoretical/
Conceptual
Framework

22
Design/
Method/
Purpose

Sample/Setting

Variables

Measurement/
Instrumentation

Data
Analysis

Results/
Findings

Patients prefer
step-by-step
assistance
Ochoa et al.
(2020)
Electronic patient
portal utilization by
neurology patient
and association
with outcomes
Country: USA
Funding: National
Center for
Advancing
Translational
Sciences of the
National Institutes
of Health under
University of
Florida
Bias: None stated

Inferred SCT

Design:
Quantitative
retrospective
cross-sectional

n= 13,483

Purpose:
Identify pt
characteristics
and care
implications
associated
with PP usage
among
neurology pts

Demographics:
PPU:
Female: 43.7%
White 44.7%
Black 27.6%
Other 43.5%
Mean age: 54.3

PPU: 5,648
NPPU: 7,853

NPPU:
Female: 56.3%
White: 55.3%
Black: 72.4%
Non-Hispanic 42.6%
Hispanic: 38.7%
Mean age: 55.6
Setting: Chart
reviews

IV:
Demographics
Neurological
Diagnosis
Number of
prescriptions
Number of
clinic visits
DV:
PPU
NPPU

Chart Reviews
PPU identified as pt
who logged into PP

Multivariate
logistic
regression
Poisson
regression

Level of
Evidence;
Application to
practice/
Generalization

CI 95%

LOE VI

Females most
likely to be
PPU

Strength large
sample size

Younger age
more likely to
be PPU

Weakness Not
generalizable
Harm: none

Hispanic and
Black
decreased odds
of PPU

Feasibility: this type
of data can be
extracted from many
types of EHRs

PPU utilized
clinic more

Utility to PICOT
This study highlights
the characteristics of
pts that are more
likely to be PPU,
thus likely more
engaged in care

Black and
Hispanic more
likely to be
hospitalized

IC: 18 years and
older, pts of
University of Florida
Health Neurology
Key: CG- control group DS- database searched DV- dependent variable EC- exclusion criteria EHR- electronic health record HBM- health belief model IC- inclusion criteria IVindependent variable LOE- level of evidence n- number of participants N- number of studies NPPU- non patient portal user OP- outpatient PP- patient portal PPU- patient portal
user pt- patient RCT-randomized clinical trial SCT- Social Cognitive Theory tx- transplants UCLA- University of California Lost Angeles UNOS- United Network for Organ
Sharing

PATIENT PORTAL
Citation

Szilagyi et al.
(2020)
Effect of patient
portal reminders
sent by a health
care system on
influenza
vaccination rates:
A randomized
clinical trial

Theoretical/
Conceptual
Framework

HBM
Principles of
Health
Literacy

Country U.S.A
Funding
UCLA Health
System

23
Design/
Method/
Purpose

Design 4-arm,
pragmatic,
intention-totreat RCT
Purpose
Evaluate
effect of
patient
reminders
send via
patient portal
on influenza
vaccination
rates

Sample/Setting

Clinics between
7/2016-7/2017
n= 164,205
Demographics
Female: 58.3%
Mean age: 46.2
Race: 57.3% White
Setting
All 57 UCLA health
system primary care
practices

Variables

Measurement/
Instrumentation

Data
Analysis

Results/
Findings

Level of
Evidence;
Application to
practice/
Generalization

DV Receive
influence
vaccine

Influenza vaccine
administration
documented in EHR
Self-reported
administration of
influenza vaccine at
outside facility

Fixed effect
Poisson
regression

CG 37.5%

LOE: II

IV1 38.0%
P= .008
CI 95%
(1.000-1.03)
ARR = 1.02

Strengths
Randomization,
large sample,

IV Patient
portal
reminders
(1,2,3)

Random
practice effects

IV2 38.2%
P= .03
CI 95% (1.001.04)
ARR=1.02

CG no patient
portal reminder

EC Not an active
portal user, a family
household member
already participating

IV3 38.2%
P= .02
CI 95%
(1.00-1.04)
ARR= 1.02

Bias
Study included
vaccines received
outside of the
UCLA Health
System
All information
was in English
Wedd et al. (2019)

Reminders
increased
vaccines
received
Inferred SCT

Design:
Quantitative

n= 710
PPU: 375
NPPU: 335

IV:
Gender, Race,
Marital Status,

Chart reviews

Compared
PPU and
NPPU using

PPU higher in
kidney tx pts

Weakness Lacks
generalizability,
limited to English
speaking only, could
not track all of
participants
vaccinated at outside
facility
Feasibility low cost,
low risk, potential
effectiveness
Utility to PICOT
PPU engage more in
preventative care
such as receiving
influenza vaccines

LOE VI

Key: CG- control group DS- database searched DV- dependent variable EC- exclusion criteria EHR- electronic health record HBM- health belief model IC- inclusion criteria IVindependent variable LOE- level of evidence n- number of participants N- number of studies NPPU- non patient portal user OP- outpatient PP- patient portal PPU- patient portal
user pt- patient RCT-randomized clinical trial SCT- Social Cognitive Theory tx- transplants UCLA- University of California Lost Angeles UNOS- United Network for Organ
Sharing

PATIENT PORTAL
Citation

Racial, ethnic, and
socioeconomic
disparities in webbased patient portal
usage among
kidney and liver
transplant
recipients: cross
sectional study
Country
USA
Funding
National Institutes
of Health grant
number
R21NR014544
Bias: None
declared

Theoretical/
Conceptual
Framework

24
Design/
Method/
Purpose

Sample/Setting

Variables

Measurement/
Instrumentation

Data
Analysis

Results/
Findings

retrospective
cross-sectional

Kidney tx: 455
Liver tx: 255

UNOS database
review

Purpose:
Examine
demographic,
clinical and
socioeconomi
c
characteristics
associated
with usage of
PP among
diverse
populations of
tx pts

Demographics
Mean Age PPU: 50.6
Mean Age NPPU:
52.1
Kidney tx:
Mean age: 49.1
Male: 55.2%
White: 33.9%
Black: 56.9%
Married: 53.4%

Education,
Insurance,
Poverty level
Kidney tx
Renal tx

chi-square
tests and
independent t
tests

PPU higher in
younger age
pts

Liver tx:
Mean age: 53.4
Male: 59.6%
White: 71.8%
Married: 65.5%
Setting: Chart
Reviews
Inclusion: kidney or
liver tx pt recipient
in Southeast between
3/2014 and 11/2016.

DV:
PPU
NPPU

Descriptive
statistics for
frequency

PPU higher in
college
graduates than
grade school
education
PPU higher in
private
insurance
compared to
public
insurance
PPU higher in
employed
compared to
unemployed

Level of
Evidence;
Application to
practice/
Generalization
Strength large
sample size

Weakness Not
generalizable
Harm: none
Feasibility: this type
of data can be
extracted from many
types of EHRs
Utility to PICOT
This study highlights
the characteristics of
pts that are more
likely to be PPU,
thus likely more
engaged in care

PPU lower in
lower poverty
levels

Key: CG- control group DS- database searched DV- dependent variable EC- exclusion criteria EHR- electronic health record HBM- health belief model IC- inclusion criteria IVindependent variable LOE- level of evidence n- number of participants N- number of studies NPPU- non patient portal user OP- outpatient PP- patient portal PPU- patient portal
user pt- patient RCT-randomized clinical trial SCT- Social Cognitive Theory tx- transplants UCLA- University of California Lost Angeles UNOS- United Network for Organ
Sharing

PATIENT PORTAL

25

Table A2
Evaluation Table of Qualitative Studies
Citation

Theoretical/
Conceptual
Framework

Design/
Method/
Sampling

Hefner et al.
(2019)

Inferred HBM

Design:
Qualitative
exploratory study

Patient portal
messaging for
care
coordination: a
qualitative
study of
perspectives of
experienced
users with
chronic
conditions
Country
USA
Funding Grant
from Ohio State
University
Department of
Family
Medicine
Bias None
evident

Purpose: To
understand how
pts with chronic
conditions utilize
the SM
component of PP

Sample/Setting Major
Variables/
Research
Questions
n = 17
1 5-person FG, 2
6-person FG

IC- current user
of MyChart PP,
diagnosed with
cardiopulmonary
condition, patient
of Department of
Family Medicine
Clinic
Setting: FG
conducted at a
round table while
eating lunch

Guided
questions
regarding:
-How pts learned
to use PP
- Any training in
use of PP
- Frequency of
SM
- Perceived value
of SM

Measurement/
Instrumentation

Data Analysis

Findings/
Themes

3 FGs conducted
by principal
investigator

FG were audio
recorded and
transcribed by 2
medical
students

Pts use SM
because it is
quicker
communication
with provider

Codebook
developed to
identify themes

SM used to
coordinate care
with different
specialties

Transcript
reviewed by 2+
code team
members

Challenges to
SM: DL, worry
about bothering
provider,

Level of Evidence;
Application to
practice/
Generalization
LOE: VI

Strength investigator
was very experienced
in FG interviews
Weakness
Only included pts
already using PP,
small sample size, no
demographic
information provided
Harm none
Feasibility may be
difficult to find a
group of pts already
using PP. Focus
group questions were
included and study
could perhaps be
replicated
Utility to PICOT
Study aim
demonstrates how pts
engage using PP

Key: CG- control group DL- digital literacy DS- database searched DV- dependent variable EC- exclusion criteria EHR- electronic health record FG- focus group HBM- health
belief model HL- health literacy IC- inclusion criteria IV- independent variable LOE- level of evidence n- number of participants N- number of studies NPPU- non patient portal
user OP- outpatient PP- patient portal PPU- patient portal user PRISMA- Preferred Reporting Items for Systematic reviews and Meta-Analyses PRISM- Practical, Robust
Implementation and Sustainability Model pt- patient REAIM- reach, effectiveness, adoption, implementation and maintenance SCT- Social Cognitive Theory SM- secure
messaging

PATIENT PORTAL

26

Citation

Theoretical/
Conceptual
Framework

Design/
Method/
Sampling

Hoogenbosch
et al. (2018)
Use and the
users of a
patient portal:
Cross-sectional
study

Unified Theory
of Acceptance
and Use of
Technology

Design: Crosssectional study

Country
Netherlands
Funding
University
Medical Centre
Utrecht
Bias None
declared

Purpose: Explore
the prevalence of
PP use and the
characteristics of
pts who use and
do not use PP

Sample/Setting Major
Variables/
Research
Questions
n= 439
PPU: 141
NPPU: 298

IC: >18 years
old, visiting on
OP department
of specified
medical center,
fluent in Dutch,
EC: inpatient
Setting: Patients
visiting OP
departments
Demographics:
Mean age PPU:
50
Mean age NPPU
aware: 53
Mean age NPPU
not aware: 55
Females 51.2%
Female PPU:
55.3%
Female NPPU
aware: 50.7%
Female NPPU
not aware: 48.4%

IV:
Gender
Chronically Il
Life Status
Education Level
eHL

DV:
PPU
NPPU but aware
of PP
NPPU not aware
of PP

Measurement/
Instrumentation

Data Analysis

Findings/
Themes

Structured paper
questionnaire

Descriptive
statistics to
identify:
PPU
NPPU but
aware of PP
NPPU not
aware of PP

PPU: 32.1%
NPPU aware of
PP: 31/2%
NPPU not
aware of PP:
36.7%

5-point Likert
scale
eHL questionnaire

Logistic
regression to
explore use of
PP

PPU were
significantly
younger, not
retired, more
often
chronically ill,
higher eHL
scale score,
more likely to
be satisfied with
care

Level of Evidence;
Application to
practice/
Generalization
LOE VI

Strength Large
sample size,
Weakness
Convenience sample
limits generalizability
Dutch eHL
questionnaire not
reliable
Harm: none
Feasibility: Similar
study could be
completed in other
OP setting
Utility to PICOT
This study highlights
the characteristics of
pts that are more
likely to be PPU, thus
likely more engaged
in care

Key: CG- control group DL- digital literacy DS- database searched DV- dependent variable EC- exclusion criteria EHR- electronic health record FG- focus group HBM- health
belief model HL- health literacy IC- inclusion criteria IV- independent variable LOE- level of evidence n- number of participants N- number of studies NPPU- non patient portal
user OP- outpatient PP- patient portal PPU- patient portal user PRISMA- Preferred Reporting Items for Systematic reviews and Meta-Analyses PRISM- Practical, Robust
Implementation and Sustainability Model pt- patient REAIM- reach, effectiveness, adoption, implementation and maintenance SCT- Social Cognitive Theory SM- secure
messaging

PATIENT PORTAL

27

Citation

Theoretical/
Conceptual
Framework

Design/
Method/
Sampling

Lyles et al.
(2020)
Using
electronic
health record
portals to
improve patient
engagement:
Research
priorities and
best practices

Principles of
implementation
science

Design:
Qualitative
Grounded Theory

Country
USA
Funding
Many grants
Bias None
evident

Purpose: To
examine studies
that address
patient
engagement with
EHR to identify
best practices and
research priorities

Sample/Setting Major
Variables/
Research
Questions
N: 52 articles

DBS: PubMed
IC: Published
between 9/20136/2019, include
intervention
EC: non-original
data- reviews,
protocols, studies
outside the USA,
observational
studies without
intervention

PP use and
outcome studies
PP usability
studies
PP barriers to
use

Measurement/
Instrumentation

Data Analysis

Findings/
Themes

Two authors
completed data
extraction and
conducted full-text
review to select
articles that met
inclusion criteria

REAIM
framework

Patient’s DL
and HL should
be examined
when
implementing
PP usage uptake
efforts

Another author
reviewed full text
of selected articles
to ensure all were
reliable

Barriers to use
should be
addressed such
as language and
access
Interventions to
promote usage
should include
in-person
tutorials
Interventions
via EHR should
be user friendly
to effect
engagement

Level of Evidence;
Application to
practice/
Generalization
LOE V

Strength
Included 52 primary
research articles
Weakness
Lower level of
evidence, only used
one database
Harm none
Feasibility
Findings could
potentially be
addressed in an
intervention
Utility to PICOT
This study highlights
the characteristics of
pts that are more
likely to be NPPU,
perhaps revealing pts
less likely to be
engaged
LOE VI

Portz et al.
Technology
Design:
n= 24
PPU:
Semi-structured
FC audio
PPU mostly
(2019)
acceptance
Qualitative
PPU: 14
Why did you
FGs
recorded and
used PP for
Using the
model
descriptive study
NPPU: 9
enroll in PP?
transcribed
communicating Strength Included
technology
NPPU
What features do Technology
verbatim
with provider
Purpose: To use
IC: Presence of
utilization survey
acceptance
you use most?
the technology
multiple chronic
Weakness
model to
Key: CG- control group DL- digital literacy DS- database searched DV- dependent variable EC- exclusion criteria EHR- electronic health record FG- focus group HBM- health
belief model HL- health literacy IC- inclusion criteria IV- independent variable LOE- level of evidence n- number of participants N- number of studies NPPU- non patient portal
user OP- outpatient PP- patient portal PPU- patient portal user PRISMA- Preferred Reporting Items for Systematic reviews and Meta-Analyses PRISM- Practical, Robust
Implementation and Sustainability Model pt- patient REAIM- reach, effectiveness, adoption, implementation and maintenance SCT- Social Cognitive Theory SM- secure
messaging

PATIENT PORTAL
Citation

explore user
experience,
intent to use,
and use
behavior of a
patient portal
among older
adults with
multiple
chronic
conditions:
Descriptive
qualitative
study
Country:
USA
Funding:
National
Institute on
Aging
Bias: None
declared

Theoretical/
Conceptual
Framework

28
Design/
Method/
Sampling
acceptance model
do describe PP
user interface and
PP user
experience, intent
to use, and use
behaviors among
older patients
with multiple
chronic conditions

Sample/Setting Major
Variables/
Research
Questions
conditions, age
>65, pt of
geriatric Denver
clinics

EC: non-English
speaker, skilled
nursing facility
residents,
dementia
diagnosis
Demographics:
Female PPU:
12(80%)
Female NPPU:
5(56%)
Mean age: 78.41
Mean age PPU:
76.4
Mean age NPPU:
82.7
White: 19
Hispanic: 3
College Grad: 9
Some College: 0
High school
Grad: 6
Income
>$30,000: 4
$300,00049,999: 13
>$50,000: 7

NPPU: Are you
interested in PP?
Are there
reasons why you
do no use PP?

Measurement/
Instrumentation

Data Analysis

Findings/
Themes

Coded with
Technology
acceptance
model driven
codes

PP users felt the
PP was user
friendly except
the registration
part
NPPU reported
PP looked to
difficult to use
and font size
was too small
PPU and NPPU
users reported
anxiety related
to computers.
PPU shared
there is a
learning curve
to use
PPU thought
the greatest
benefit was
communicating
with provider.
NPPU preferred
telephone
communications

Level of Evidence;
Application to
practice/
Generalization
Small sample size,
not generalizable
especially to less
educated or lower
income.
Harm: none
Feasibility
FG questions
provided, could be
replicated with PP in
another setting.
However,
functionality may
vary with other PP
systems
Utility to PICOT
This study highlights
the characteristics of
pts that are more
likely to be NPPU,
perhaps revealing pts
less likely to be
engaged

Key: CG- control group DL- digital literacy DS- database searched DV- dependent variable EC- exclusion criteria EHR- electronic health record FG- focus group HBM- health
belief model HL- health literacy IC- inclusion criteria IV- independent variable LOE- level of evidence n- number of participants N- number of studies NPPU- non patient portal
user OP- outpatient PP- patient portal PPU- patient portal user PRISMA- Preferred Reporting Items for Systematic reviews and Meta-Analyses PRISM- Practical, Robust
Implementation and Sustainability Model pt- patient REAIM- reach, effectiveness, adoption, implementation and maintenance SCT- Social Cognitive Theory SM- secure
messaging

PATIENT PORTAL

29

Citation

Theoretical/
Conceptual
Framework

Design/
Method/
Sampling

Risling et al.
(2018)
Defining
empowerment
and supporting
engagement
using the
patient views
from the citizen
health
information
portal:
Qualitative
study

Roger’s
diffusion of
innovations
theory

Qualitative,
interpretive
description

Country:
Canada
Funding:
University of
Saskatchewan
Bias: None
listed

Purpose: explore
participants view
on PP usage and
feeling of
empowerment

Sample/Setting Major
Variables/
Research
Questions
N= 26

Demographics
Female= 18
Male= 8
Majority age 6069
Majority of
participants had a
chronic illness
Setting
Residents of
Western
Canadian
province using
PP
Inclusion
English speaking
Participants of
the Citizen
Health
Information
Portal

Themes of
Empowerment
Identified:
Being heard:
-Knowing moreaccess to more
information
-Seeing what
they (the
provider) seeaccess to health
data
Moving forward:
-Owning future
steps- engaging
in self-care
behaviors
-Promoting
future caredesire to use PP

Measurement/
Instrumentation

Data Analysis

Findings/
Themes

Semi structured
interviews

Line-by-line
coding of data
within
transcription of
interview
Explore
commonalities
and differences
Reflective
memos

PP influenced
patient
empowerment
by making
participants feel
like they know
more with the
information
provided in PP
and by allowing
the patient to
see what
providers see
such as lab
results
PP had no clear
impact on
clinical
outcomes
Improving
relationship
with provider
improved
engagement

Level of Evidence;
Application to
practice/
Generalization
LOE VI

Strengths identifies
gap in literature
exploring
empowerment
Weakness
Specific to Citizen
Health Information
Portal
No comparison to
non-PP users
Small sample size
Harm: none
Feasibility
Interview questions
not explicitly states,
would be difficult to
replicate
Utility to PICOT
Directly related to
intervention, pts who
are PPU feel more
empowered/engaged

Stewart et al.
Framework for
Design:
n= 40
How do online
Semi structured
Interviews were PP impact on
LOE: VI
(2020)
patient
Qualitative,
PP support
phone interviews
recorded and
patientThe promise of
Demographics
engagement
grounded theory
patient
transcribed
healthcare team Strength:
Men: 32 (80%)
Participants felt very
patient portals
engagement for
relationship
Key: CG- control group DL- digital literacy DS- database searched DV- dependent variable EC- exclusion criteria EHR- electronic health record FG- focus group HBM- health
belief model HL- health literacy IC- inclusion criteria IV- independent variable LOE- level of evidence n- number of participants N- number of studies NPPU- non patient portal
user OP- outpatient PP- patient portal PPU- patient portal user PRISMA- Preferred Reporting Items for Systematic reviews and Meta-Analyses PRISM- Practical, Robust
Implementation and Sustainability Model pt- patient REAIM- reach, effectiveness, adoption, implementation and maintenance SCT- Social Cognitive Theory SM- secure
messaging

PATIENT PORTAL
Citation

for individuals
living with
chronic illness:
Qualitative
study
identifying
pathways of
patient
engagement
Country:
USA
Funding:
Center for
Healthcare
Organization
and
Implementation
Research
Bias: None
listed

Theoretical/
Conceptual
Framework

30
Design/
Method/
Sampling
Purpose: Explore
how PP facilitate
engagement in
patients with
diabetes

Sample/Setting Major
Variables/
Research
Questions
Ethnicity: 33
(85%) white
Mean age: 65.9

Setting
Phone interviews
Inclusion
Veterans who
use PP My
HealtheVet
Respond to
survey inquiring
if interested in
participating
Diabetes
Exclusion
No mental health
illness

individuals
living with
diabetes?

Measurement/
Instrumentation

Data Analysis

Findings/
Themes

Double-coded
for multiple
themes by
several coders
Coded of
deducting and
inductive
themes

Feelings of
empowerment
with PP use
PP for
collaboration
PP and care
plan changes
PP
communication
for clarification
PP messaging
challenges
PP med refill
challenges
PP has no effect
on clinical
outcomes
Inferred
improved
patient
satisfaction with
PP usage
Provider
feedback on PP
improved
engagement

Level of Evidence;
Application to
practice/
Generalization
positive about PP
usage

Weakness:
participants were
already enrolled in
My HealtheVet
PP/already engaged
Specific to veterans
with diabetes
No comparison to
non-PP users
Small sample size
Harm: none
Feasibility: Interview
questions not
explicitly states,
would be difficult to
replicate
Utility to PICOT
Directly related to
intervention, pts who
are PPU feel more
empowered/engaged

Key: CG- control group DL- digital literacy DS- database searched DV- dependent variable EC- exclusion criteria EHR- electronic health record FG- focus group HBM- health
belief model HL- health literacy IC- inclusion criteria IV- independent variable LOE- level of evidence n- number of participants N- number of studies NPPU- non patient portal
user OP- outpatient PP- patient portal PPU- patient portal user PRISMA- Preferred Reporting Items for Systematic reviews and Meta-Analyses PRISM- Practical, Robust
Implementation and Sustainability Model pt- patient REAIM- reach, effectiveness, adoption, implementation and maintenance SCT- Social Cognitive Theory SM- secure
messaging

PATIENT PORTAL

31

Table A3
Synthesis Table
Hefner et
al.
2019
VI
QL/ ES

Hoogenbosch
et al.
2018
V
QL/ CSS

Lyles et al.

Framework
No Evident
Bias

HBM
X

UTA
X

USA
Netherlands
Canada

X

Size

17

Author
Year
LOE
Design

Portz et al.

X

X

2020
V
QT/ GT

2019
VI
QL/ DS

Risling et
al.
2018
VI
QL/ ID

Stewart et
al.
2020
VI
QL/ GT

Szilagyi et
al.
2020
II
QT/RCT

Wedd et
al.
2019
VI
QT/ RP
CSS
SCT
X

SCT
X

TAM
X

RDI
X

FPE
X

HBM

X

X

X

X

X

X
X
439
PPU: 141
NPPU: 298

Female (%)

PPU: 55.3
NPPU: 49.3

Mean Age

PPU: 50
NPPU: 54

PP Reminders
Chart Reviews

Nahm et
al.
2017
II
RCT

PIS
X

Ochoa et
al.
2020
VI
QT/ RP
CSS
SCT
X

52 articles

13,483
PPU: 5,648
NPPU:
7,853
PPU: 43.7
NPPU:
56.3
PPU: 54.3
NPPU:
55.6

X

74

24
PPU: 14
NPPU: 9

26

40

164,205

710
PPU: 375
NPPU: 335

71.6

PPU: 80
NPPU: 56

69.2

20

58.3%

57.4

PPU: 76.4
NPPU:
82.7

60-69 age

65.9

46.2

PPU: 23.5
NPPU:
19.7
PPU: 50.6
NPPU:
52.1

provided in
categories

X
X

X

Key: CSS- cross sectional study DL- digital literacy DS- descriptive study DV- dependent variable ES- exploratory study FG- focus group FPE-framework for patient
engagement FTF- face-to-face GT- grounded theory HBM- health belief model HL- health literacy ID- interpretive description LOE- level of evidence NPPU- non patient portal
user PIS- Principles of implementation science PP- patient portal PPU- patient portal user QL- qualitative QT- quantitative RDI- Roger’s diffusion of innovations theory RPretrospective SCT- Social Cognitive Theory SM- secure messaging TAM- technology acceptance model tx- transplants UNOS- United Network for Organ Sharing UTA- Unified
theory of acceptance and use of technology ↑- increased ↓- decreased ▲- more likely to utilize patient portal▼- less likely to utilize patient portal ▬- no effect

PATIENT PORTAL
Questionnaire
Interviews
Full-text
Review
FG
FTF Education
Communication
Younger Age
Minorities
Insured
Higher HL
Lower DL
Chronically Ill
Vaccine Uptake
Engagement
Satisfaction
Empowerment
Clinical
Outcomes

32
X

X

X
X

X

X
X

X
X
↑

↑
▲

▲
▼

▲
▼
▲
▼

▲

▲

▲
▲
▼
▲
↑

▼

▼

▼
▲
↑

↑

↑

↑
↑
▬

↑
↑
↑
▬

▬

Key: CSS- cross sectional study DL- digital literacy DS- descriptive study DV- dependent variable ES- exploratory study FG- focus group FPE-framework for patient
engagement FTF- face-to-face GT- grounded theory HBM- health belief model HL- health literacy ID- interpretive description LOE- level of evidence NPPU- non patient portal
user PIS- Principles of implementation science PP- patient portal PPU- patient portal user QL- qualitative QT- quantitative RDI- Roger’s diffusion of innovations theory RPretrospective SCT- Social Cognitive Theory SM- secure messaging TAM- technology acceptance model tx- transplants UNOS- United Network for Organ Sharing UTA- Unified
theory of acceptance and use of technology ↑- increased ↓- decreased ▲- more likely to utilize patient portal▼- less likely to utilize patient portal ▬- no effect

PATIENT PORTAL

33
Appendix B
Theoretical Framework Models

Figure B1
Social Cognitive Theory

(Health Communication Capacity Collaborative, n. d.)

PATIENT PORTAL

34

Figure B2
Rosswurm and Larrabee Model

(Rosswurm & Larrabee, 1999)

PATIENT PORTAL

35
Appendix C
Budget Table

Phase

Activities

Preparation

Create a video of benefits of
patient portal use and
instructions on how to use
portal
30 minutes spent with Medical
Director/Site Champion to
evaluate what she may want to
be included in the educational
video. Time spent away from
patient care/ clinic duties
Create Quick Response (QR)
code to link with video
Design and print 400
brochures with QR code on it
($0.14 per page)
No loss of productivity
estimated
SPSS Software

Delivery
Evaluation
Total
Funding

Cost

subtotal

Total

$0

$0

$0

$0

$0

$0

$0

$0

$0

$0.14

$56

$60

$0

$0

$0

$47

$47

$47
$107

Project Facilitator