IMPLEMENTATION OF A SUICIDAL IDEATION TREATMENT

Implementation of a Suicidal Ideation Treatment Algorithm in a Military Medicine
Primary Care Clinic

Mianna Gale
Edson College of Nursing and Health Innovation Arizona State University

Author Note
Mianna Gale is a full-time student at Arizona State University.
There are no known conflicts of interest to disclose.
Correspondence concerning this article should be addressed to Mianna Gale, Edson
College of Nursing, Arizona State University, 550 N. 3rd Street, Phoenix, AZ 85004. Email:
mmcarls1@asu.edu.

1

IMPLEMENTATION OF A SUICIDAL IDEATION TREATMENT

2

Abstract
Primary care providers (PCPs) are frequently the first line of treatment for suicidal ideation (SI)
patients. Many PCPs report low self-efficacy in treating suicidal patients, leading to
inappropriate treatment plans or avoidance of discussing SI. This quality improvement project
based on the Uncertainty Reduction theory aimed to evaluate PCP's perceptions of an SI
treatment algorithm and its impact on self-efficacy. Secondary aims included assessing PCP's
confidence in treating suicidal patients and current treatment practices. A pre- then postintervention survey design was utilized. All PCPs treating patients in a military medicine clinic
were invited to participate in the project. Participants were sent a recruitment email containing
the suicidal ideation treatment algorithm and a link to a survey developed with Qualtrics
software. Participants were asked to review the SI algorithm, answer the baseline survey
questions, and complete a second eight-week survey. For human subjects' protection, the survey
responses were anonymous. Demographic data collected included years of clinical experience
and licensure type. The data were evaluated with Intellectus software. Due to limited
participation, N=4, there was insufficient data to determine the significance of implementing the
SI algorithm in a primary care clinic. Central tendencies showed that most providers (n=3,
75.00%) felt less than confident treating suicidal patients. Half of the providers asked non-mental
health patients about suicide less than 40% of the time (n=2, 50.00%). The data suggest that
PCPs feel uncomfortable treating suicidal patients and may benefit from additional resources and
training in this area.
Keywords: primary care, suicidal ideation treatment algorithm, provider self-confidence

IMPLEMENTATION OF A SUICIDAL IDEATION TREATMENT

3

Implementation of a Suicidal Ideation Treatment Algorithm in a Military Medicine
Primary Care Clinic
Suicide rates nationally and within the military continue to increase every year despite
multiple local, state, and national initiatives to raise suicide awareness and prevention over the
last several decades. The lack of impact from these initiatives may be related to a lack of
provider confidence in treating this patient population and the continued stigma associated with
suicide and suicidal ideations (SI) in the general public and within the healthcare community.
Background and Significance
There is a significant under-reporting of SI in patients seen by a primary care provider
(PCP). In a group of people who completed suicide, half were seen within a month, and almost
80% were seen within a year of their deaths by a PCP (Ahmedani et al., 2014; Hauge et al.,
2018; Rodi et al., 2009). Two-thirds of patients screened for SI before their deaths denied having
them, and half of those patients were dead within two days of being screened (Berman, 2017).
These reports indicate a substantial number of patients who could have received lifesaving
intervention at a primary care office before their deaths.
PCPs frequently report low self-efficacy when treating suicidal patients, likely leading to
less effective suicidal ideation assessment. Most providers indicate they feel confident in
diagnosing depression, but when asked about suicidal patients, 60% of providers report feeling
uncomfortable with managing their care (Michail et al., 2017). Due to this uncertainty,
physicians report they only ask patients about SI approximately 37% of the time. This percentage
remains the same even when the provider perceives the patient is depressed (Hooper et al.,
2012). These statistics are especially concerning given the fact that suicidal patients are
significantly more likely to be seen by a PCP than specialty care (John et al., 2020; Schaffer et

IMPLEMENTATION OF A SUICIDAL IDEATION TREATMENT

4

al., 2016). This lack of self-efficacy is likely related to the stigma associated with SI patients and
the ambiguity of SI, considering that the severity of suicidality is entirely based on subjective
information disclosed by the patient. Providers fear financial liability, personal guilt, and are at
an increased risk of experiencing burnout following a patient's suicide (Oravecz & Moore, 2006;
Schmitz et al., 2012).
Purpose and Rationale
Suicide is preventable, and appropriate risk assessments and interventions save lives
(Hogan & Grumet, 2016). The actions that occur following a patient's self-report of SI impact
their potential outcomes and the likelihood of reporting suicidal thoughts in the future. An
underreaction on the provider's part can lead to a suicide attempt. In contrast, an overreaction can
lead to increased depression, isolation, hopelessness, stigmatization, financial strain, and a
decreased likelihood of patient disclosure in the future. Patients have reported they frequently do
not disclose thoughts of suicide because of fears of overreaction on the provider's part (Richards
et al., 2019).
The level of importance for developing an appropriate treatment plan, the high number of
patients that completed suicide despite being seen by a PCP shortly before their deaths, and the
reported general lack of confidence among PCPs in treating SI indicate a treatment gap that
needs to be addressed. The purpose of this project was to evaluate PCP's perceptions of a SI
treatment algorithm and its impact on self-efficacy. The secondary aim was to assess PCP's
current confidence in treating SI and current treatment practices.
Epidemiological Data
Suicide is devastating for the friends and family of the deceased and has a significant
impact on society as a whole. In the United States (US), suicide is the 10th leading cause of

IMPLEMENTATION OF A SUICIDAL IDEATION TREATMENT

5

death in all age groups, the second leading cause of death in adolescents and young adults, and
the fourth leading cause of death in middle-aged adults (Curtin, 2020; Hedegaard et al., 2021).
The total societal monetary cost of suicide in medical expenses, lost productivity, and resource
utilization for combined suicide attempts and completions in 2013 was 93.5 billion dollars
(Center for Disease Control and Prevention [CDC], 2017).
In 2019 there were 1.4 million suicide attempts and over 47,000 completed suicides
throughout the US (American Foundation for Suicide Prevention [AFSP], 2021). That same year
almost 5% of adults and 19% of high school-aged adolescents reported serious thoughts about
committing suicide (America's Health Rankings, 2021; National Institute of Mental Health
[NIMH], 2021). The global pandemic has increased suicide rates, indicating urgency in
addressing this issue. Post-COVID ER visits in the young adult population, ages 12-25, have
increased by 30-50% (Center for Disease Control and Prevention [CDC], 2021). The rate of
active-duty military member suicides increased by 25% in 2020 compared to the previous year
(Losey, 2021).
Internal Evidence
The project site is a military primary care clinic in the southwest United States that treats
active-duty military personnel, military retirees, and their families. The age range of patients is
birth to death. Approximately 90,000 primary care visits occur in the clinic every year (Luke Air
Force Base, n.d.). Routine Patient Health Questionaire-9 and Generalized Anxiety Disorder-7
screenings are performed on all patients being seen in the 56th Medical Group. There is no
specific treatment tool in place that guides the care of patients after they have reported SI. The
clinic's internal evidence indicates a general lack of confidence in treating SI and a lack of
available resources to help guide treatment. Providers within the clinic report that when a patient

IMPLEMENTATION OF A SUICIDAL IDEATION TREATMENT

6

discloses thoughts of suicide, they are unsure of the next appropriate steps. They also have
difficulty differentiating between appropriate patients to continue in the clinic and those that
require a higher level of care (H. Toberman, personal communication, February 23, 2021).
The United States Air Force (USAF) has maintained a strong focus on suicide prevention
since implementing the Suicide Prevention Initiative in the 1990s. This initiative includes 11
components that focus on prevention, intervention, and post-suicide response (The United States
Air Force, 2019). The main focus of this initiative is to influence culture throughout the USAF
by providing education and guidance to Unit Commanders, leadership, individual Airmen, and
military families. The initiative effectively decreased suicide rates by approximately 30% within
the first few years of implementation (Hogan & Grumet, 2016). However, this initiative focuses
on influencing USAF culture as a whole and does not provide guidance for the medical treatment
of SI patients.
PICOT Question
Researching this treatment gap led to the PICO question, [P] in primary care providers in
a military medicine clinic [I] how does having a suicidal ideation treatment algorithm [C]
compared to not having a treatment algorithm [O] affect the provider's level of treatment
confidence?
Evidence Synthesis
Search Strategy and Literature Review
A thorough review of the evidence was conducted to evaluate this PICO question. This
review included four databases: PsychInfo, PubMed, Academic Search Premier, and the
Cochrane library. PsychInfo was incorporated due to the intervention's mental health aspect, and
the Cochrane Library was chosen to encompass relevant grey literature. PubMed and Academic

IMPLEMENTATION OF A SUICIDAL IDEATION TREATMENT

7

Search Premier offered a broad range of available literature. The initial search incorporated key
terms for all of the components in the PICO question, including relevant synonyms: primary care
providers, primary healthcare, suicidal ideation treatment algorithm, provider confidence, and
self-efficacy. However, using these specific terms resulted in eight articles among all four
databases.
As a result, the search was widened to include all treatment algorithms utilized in primary
care by removing the words suicidal ideation from the key terms. Additional MeSH terms were
also added to capture more material resulting in a search with the following key terms: primary
care, primary health care, general practice, algorithm, treatment algorithm, intervention
guideline, treatment protocol, provider confidence, self-efficacy, attitude, and perception. These
terms resulted in a much higher yield; 42,630 results from PsychInfo, 431 results from PubMed,
526 results from Academic Search Premier, and 61 from the Cochrane library. A considerable
proportion of the articles were irrelevant to the PICO question due to the common use of the
word algorithm in explaining methods for statistical data collection. Therefore, the search was
narrowed by incorporating the keyword implementation and only included articles with
publication dates from 2001 until the present to elicit a more relevant yield and narrowed. With
this strategy, the final results included one from PsychInfo, 60 results from PubMed, 138 results
from Academic Search Premier, and 19 results from the Cochrane library. These results were
evaluated by reading the title and abstract of each article to determine relevance to the project.
This process resulted in a final yield of 22 relevant articles.
The quality of studies available to evaluate provider perceptions and confidence were
low-level evidence. Therefore patient outcomes were also included in the evaluation to
determine how algorithms affected provider efficiency as evidenced by improved patient

IMPLEMENTATION OF A SUICIDAL IDEATION TREATMENT

8

outcomes. Due to the varying levels of evidence associated with the different outcomes, several
articles that evaluated physiologic medical problems were chosen, in addition to those that
evaluated subjective data. The physiologic medical data generally provides a higher level of
evidence. However, mental health complaints have subjective data. This combination was
determined to be the most robust means of evaluating the effectiveness of the intervention type.
Inclusion criteria for the relevant articles included a primary care setting, the use of a
treatment algorithm or equivocal brief intervention tool, and post-intervention provider
perceptions or patient outcomes. Articles were excluded if the results from the intervention were
related to a multi-faceted program in general and did not provide specific information regarding
the treatment algorithm portion of the intervention. After determining inclusion, each of the 22
relevant articles was evaluated with a rapid critical appraisal checklist to select the ten most
relevant studies. The final studies included four randomized control trials, two non-randomized
controlled trials, and four cohort studies (See Appendix A).
Foundation of Research and Evidence
All studies showed an improvement in measured outcomes. Studies showed homogeneity
in the consistency of the primary care clinic setting and heterogeneity in the age range and
symptom type. Studies included pediatric patients, adult patients, and geriatric patients. Studies
were also conducted in multiple countries and several different socioeconomic settings (See
Appendix A). Four studies evaluated subjective symptoms, and the remainder assessed
physiologic medical conditions. A medication guide was included in four studies, along with the
algorithmic tool. The use of multiple interventions limits the ability to determine if the
algorithmic tool alone would have led to the same outcome improvements. The majority of the

IMPLEMENTATION OF A SUICIDAL IDEATION TREATMENT

9

studies were quality improvement projects. Most of the data were interviews, surveys, or chart
reviews. Changes in screening scores were utilized in two studies (See Appendix B).
The combination of study types and evaluation of multiple outcomes gave strength to the
evidence by determining that the implementation of algorithmic tools improves care in a broad
population range, a variety of settings, and in multiple disease processes. Algorithmic tools
improved patient outcomes and provider knowledge and confidence in treating patients
appropriately.
Influence on Project
Algorithms are commonly used in primary care to guide treatment. The review of
evidence revealed that research specifically focused on using SI treatment algorithms in the
primary care setting is limited. However, the evidence supports the use of general patient
treatment algorithms in primary care based on the results indicating they effectively improve
patient outcomes, provider knowledge, and provider confidence. Implementing concise and
efficient treatment algorithms improves care outcomes (Haran et al., 2020). Additionally, the
American Family Physicians website (2021) supports the use of treatment algorithms by
providing links to hundreds of algorithms that can assist in treating a plethora of care items,
including medication management, disease management, and risk assessments for mental
illnesses. Despite the lack of research specific to primary care patients with suicidal ideations, all
of the evidence indicated that incorporating a suicidal ideation treatment algorithm in the
primary care settings may impact provider confidence and patient outcomes. Therefore a
treatment algorithm was an appropriate intervention to utilize for this project.
Theoretical Framework

IMPLEMENTATION OF A SUICIDAL IDEATION TREATMENT

10

The goal of providing practitioners with a treatment algorithm was to increase selfefficacy by reducing uncertainty. The Uncertainty Reduction Theory (URT) was used as the
theoretical framework to guide this project (See Appendix C). According to the URT by Mark
Redmond (2015), people attempt to reduce uncertainty in interpersonal relationships by
increasing the amount of information they have to predict how an interaction will unfold. This
concept applies to the provider and patient relationship because providers are more comfortable
treating patients when they feel they have enough information or knowledge about the patient's
complaint to confidently predict how the patient interaction will progress and what treatment
outcomes will be. The project aimed to reduce providers' uncertainty by giving them a treatment
algorithm that provided step-by-step instructions for developing an appropriate treatment plan
for suicidal patients. Ideally, this reduction in provider uncertainty will lead to more competent
assessments and appropriate care for patients in the future.
Implementation Framework
The implementation framework for this project was a quality improvement project.
Quality improvement projects aim to adapt current processes or implement new processes within
a specific site to improve patient outcomes (Conner, 2014). The quality improvement model
utilized was the Find, Organize, Clarify, Understand, Select, Plan, Do, Study, Act (FOCUSPDSA) model (See Appendix D). This model is an expansion of the commonly utilized PDSA
model. It was appropriate for this project because it provided steps for project development and
implementation (American College of Cardiology [ACC], 2013). The PDSA model evaluates the
impact of small changes on patient outcomes and is appropriate for implementation in a small
setting such as a single clinic (Institute for Healthcare Improvement [IHI], n.d.). Incorporating

IMPLEMENTATION OF A SUICIDAL IDEATION TREATMENT

11

the additional FOCUS framework provided a step-by-step guide for developing the intervention
before implementation (Taylor et al., 2013).
Methods
Ethical Consideration and Human Subject Protection
There were no significant ethical concerns for this project. No identifying information
was collected to protect participants from a loss of anonymity, and there was no follow-up with
individual participants. The initial and follow-up surveys were not linked to each other.
Additionally, no sensitive information was collected from participants. Therefore, if anonymity
is lost, there will likely be no significant consequences for individual participants.
The Arizona State University IRB approved this project's social behavioral protocol
application with an expedited review. There is no IRB at the site organization. Therefore, no
additional approvals were required.
Population and Setting
Participants in the project were physicians, family nurse practitioners, and physician's
assistants working in a military medicine primary care facility in the Southwestern US. The
clinic treats active-duty military personnel, military retirees, and their families. The age range of
patients is birth to death. The participants were active-duty military members and Department of
Defense contract employees. Inclusion criteria included actively working in direct patient care
within the clinic. Any providers either out of the clinic for a prolonged period due to
deployments or medical leave or not currently practicing in direct patient care were excluded. All
providers that met the criteria for inclusion were recruited by email.
Project Description and Timeline

IMPLEMENTATION OF A SUICIDAL IDEATION TREATMENT

12

The suicidal ideation treatment algorithm from the VA/DoD Clinical Practice Guideline
for the Assessment and Management of Patients at Risk for Suicide was used as the intervention
for this project (The Department of Veteran Affairs [VA] & The Department of Defense [DoD],
2019). The guideline was created by an evidence-based project workgroup composed of medical
specialists in a joint effort between the Department of Veteran Affairs and the Department of
Defense (The Department of Veteran Affairs [VA] & The Department of Defense [DoD], 2013).
The guideline was initially released in 2013, and an updated version was released in 2019. The
original guideline was over 100 pages, and the revised version is 32 pages. The length of these
documents makes it impractical for use as a brief intervention tool which is likely why it is not
utilized routinely, even within the military community, despite being full of useful information
and readily available on the Department of Defense website. Therefore, only a pdf copy of the
treatment algorithm, located on pages 8-14 of the revised document, was utilized as the
intervention for this project (See Appendix E).
Recruitment and implementation for the project were conducted via email. In August
2022, an email was sent to potential participants. The email contained a description of the
project, a request for participation, a link to the initial survey, and instructions to review the
attached algorithm before answering the survey questions. The intervention was included as a
pdf attachment to the email. An email was sent eight weeks later to complete the second survey.
Instrumentation, Data Collection, and Data Analysis
Surveys created with Qualtrics software for this project were utilized for data collection.
The initial survey contained questions about providers' treatment confidence before and after
reviewing the algorithm, current practices related to discussing suicide with patients, and
participants' perceptions of the algorithm. The second survey would have evaluated change over

IMPLEMENTATION OF A SUICIDAL IDEATION TREATMENT

13

time with questions about treatment confidence and if the participants have utilized the algorithm
for patient care since they received it eight weeks ago. Demographic data included gender, type
of licensure, and years of clinical experience. Ordinal data from the Likert scale survey questions
and demographic data were extracted from the surveys and evaluated with Intellectus Statistics.
Descriptive statistical analysis was performed on the data, including central tendencies and
frequencies. There was no funding for the project, and no compensation was provided to
participants.
Results
The study participants (N=4) included two nurse practitioners, one physician, and one
physician assistant. Half of the participants had more than ten years of experience, and all
participants were employed in primary care.
Outcomes
The primary outcome of this project was to evaluate PCP's perceptions of the SI
treatment algorithm and its impact on self-efficacy. To assess the primary outcome, the
participants were asked to rank the algorithm's applicability for use in the primary care setting on
a scale of one-not applicable to five-standard of care. The majority of participants reported this
tool was applicable for use in primary care with an average of 4.67 (SD = 0.58, SEM = 0.33, Min
= 4.00, Max = 5.00). The participants were also asked the likelihood that they would use this tool
in their practice in the future on a scale of one-will not use to five-definitely will use. The
majority of participants reported they were likely to use the algorithm with an average of 4.00
(SD = 1.00, SEM = 0.58, Min = 3.00, Max = 5.00).
A comparison was going to be made between the initial and eight-week follow-up data to
assess the impact on self-efficacy. However, only one provider completed the follow-up survey,

IMPLEMENTATION OF A SUICIDAL IDEATION TREATMENT

14

and therefore there was not enough data available to compare groups. In the initial survey,
participants were asked to compare their confidence in treating patients before viewing the
algorithm compared to after viewing the algorithm. Prior to viewing the algorithm all of the
participants reported they were at least confident asking patients about SI (n=4, 100%) with an
average of 3.50 (SD = 1.00, SEM = 0.50, Min = 3.00, Max = 5.00). After viewing the algorithm
the majority of participants were very confident (n=2, 66%) and one participant reported being
somewhat confident confident (n=1, 33%) with an average of 3.33 (SD = 1.15, SEM = 0.67, Min
= 2.00, Max = 4.00).
The secondary aim was to evaluate PCP's current confidence in treating SI patients and
current treatment practices. The majority of participants reported they were less than confident
treating suicidal patients with an average of 2.25 (SD = 1.26, SEM = 0.63, Min = 1.00, Max =
4.00). When asked about current treatment practices, participants reported that they always ask
mental health patients about SI with an average of 5.00 (SD = 0.00, SEM = 0.00, Min = 5.00,
Max = 5.00). However, half of the participants ask non-mental health patients about SI rarely or
less (n=2, 50%) with an average of 3.00 (SD = 1.83, SEM = 0.91, Min = 1.00, Max = 5.00).
Project Impact
There was insufficient data to determine whether the algorithm would impact the
provider's long-term self-efficacy. Participants reported they would use this tool in their practice
in the future, indicating that they are looking for additional resources to guide the care of SI
patients, and the use of these resources may improve self-efficacy. This project can be sustained
by embedding the SI treatment algorithm into the electronic health record used in all military
medicine clinics in all of the military branches. Embedding the algorithm would allow it to be
accessed freely by all providers.

IMPLEMENTATION OF A SUICIDAL IDEATION TREATMENT

15

This process change would influence many stakeholders within the organization.
Organizational leadership is responsible for organization-wide operations, including creating,
disseminating, and implementing policies and procedures, including those that dictate patient
care. For this process change to become utilized throughout the military, organizational
leadership would need to rewrite the current policies to include the use of the algorithm in
routine patient care. Another key stakeholder is the leadership team within each specific Medical
Group. The group leadership team is responsible for the health and safety of the personnel and
patients within the Medical Group. The primary care providers are stakeholders and are directly
responsible for patient care, including the appropriate assessment, treatment, and long-term
follow-up for each patient. Patients are also stakeholders because they are the recipients of
service. Patients are directly affected by changes in care and providers' level of self-efficacy and
competence in providing them with care. The medical group leadership, primary care providers,
and patients will all be directly affected by the influence of the implementation of the algorithm.
Discussion
Providers in the clinic had a positive response to the intervention. They indicated they
would use this algorithm in their practice moving forward, and they felt it applied to the primary
care setting. Providers reporting low confidence in treating SI patients was in line with previous
studies (Michail et al., 2017). Interestingly, one provider reported feeling less confident after
viewing the algorithm than before viewing it. This may have been an error in placing the prequestion response in the post-question answer. It also could have been due to the provider
realizing after viewing the algorithm that they did not know as much about the SI assessment
process as they previously believed they did.

IMPLEMENTATION OF A SUICIDAL IDEATION TREATMENT

16

The results of current patient care practices were mixed compared to other studies.
Participants reported that they do not routinely ask non-mental health patients about SI, which is
in line with previous studies (Graham et al., 2011). However, participants reported that they
always ask mental health patients about SI. In previous studies, even mental health patients were
only asked about SI less than 40% of the time (Hooper et al., 2012). This difference may be due
to specialized training provided in this specific clinic or how the question was worded.
Limitations and Barriers
The small number of participants and the lack of long-term follow-up data were
significant limitations to this study. Several barriers were encountered during this project. The
COVID-19 pandemic limited the ability to provide in-person learning or direction at the clinic
site. Conducting the project exclusively online via email led to some confusion in completing the
survey. One participant did not review the algorithm before completing the initial survey. The
online-only format also likely decreased the number of participants for the study. The other
significant barrier was a last-minute deployment during the project. The deployment took most
participants out of the clinic before the second survey was sent. Therefore those participants
were unavailable to complete the follow-up survey.
Conclusion
Disseminating information to providers regarding the need to improve practice is not
challenging. The difficulty lies in initiating a behavior change that will lead to treatment
improvements. Given the high number of patients seen by a PCP before their deaths by suicide,
improving training and resources in this population can profoundly impact SI patients. It may
potentially be the key to reducing suicide rates in the future. Although the impact of
implementing the VA/DOD suicidal ideation treatment algorithm could not be adequately

IMPLEMENTATION OF A SUICIDAL IDEATION TREATMENT

17

assessed due to the limited data collected, the consensus amongst providers was that the
treatment algorithm was applicable in the primary care setting. Therefore the recommendation is
to make this resource available to all PCPs working in military medicine clinics throughout the
various branches of the armed forces. Further research should include large-scale longitudinal
studies that can assess the impact on provider self-efficacy and patient outcomes after the
VA/DOD suicidal ideation treatment algorithm has been implemented in military medicine
primary care clinics.

IMPLEMENTATION OF A SUICIDAL IDEATION TREATMENT

18

References
Ahmedani, B. K., Simon, G. E., Stewart, C., Beck, A., Waitzfelder, B. E., Rossom, R., Lynch, F.,
Owen-Smith, A., Hunkeler, E. M., Whiteside, U., Operskalski, B. H., Coffey, M., &
Solberg, L. I. (2014). Health care contacts in the year before suicide death. Journal of
General Internal Medicine, 29(6), 870–877. https://doi.org/10.1007/s11606-014-2767-3
Alexopoulos, G. S., Reynolds, C. F., Bruce, M. L., Katz, I. R., Raue, P. J., Mulsant, B. H., Oslin,
D. W., & Ten Have, T. (2009). Reducing suicidal ideation and depression in older
primary care patients: 24-month outcomes of the PROSPECT study. American Journal of
Psychiatry, 166(8), 882–890. https://doi.org/10.1176/appi.ajp.2009.08121779
Algorithms. (2021). American Family Physicians. Retrieved March 29, 2021, from
https://www.aafp.org/afp/algorithms/viewAll.htm
America's Health Rankings. (2021). Explore teen suicide in the United States. Retrieved March
27, 2021, from https://www.americashealthrankings.org/explore/health-of-women-andchildren/measure/teen_suicide/state/ALL
American College of Cardiology. (2013). Introduction to quality improvement and the FOCUSPDSA model. https://cvquality.acc.org/docs/default-source/qitoolkit/01_introtoqiandthefocus_pdsamodel_12-10-13new.pdf?sfvrsn=44478fbf_2
American Foundation for Suicide Prevention. (2021, February 9). Suicide statistics.
https://afsp.org/suicide-statistics/
Amin, R., & Thomas, M. (2020). Effects of brief depression and anxiety management training on
a US Army division’s primary care providers. Military Medicine, 185(5-6), e719–e723.
https://doi.org/10.1093/milmed/usz443

IMPLEMENTATION OF A SUICIDAL IDEATION TREATMENT

19

Aminsharifi, A., Schulman, A., Anderson, J., Fish, L., Oeffinger, K., Shah, K., Sze, C., Tay, K.
J., Tsivian, E., & Polascik, T. J. (2018). Primary care perspective and implementation of
a multidisciplinary, institutional prostate cancer screening algorithm embedded in the
electronic health record. Urologic Oncology: Seminars and Original Investigations,
36(11), 502.e1–502.e6. https://doi.org/10.1016/j.urolonc.2018.07.016
Berman, A. L. (2017). Risk factors proximate to suicide and suicide risk assessment in the
context of denied suicide ideation. Suicide and Life-Threatening Behavior, 48(3), 340–
352. https://doi.org/10.1111/sltb.12351
Bernhardsson, S., Larsson, M., Eggertsen, R., Olsén, M., Johansson, K., Nilsen, P., Nordeman,
L., van Tulder, M., & Öberg, B. (2014). Evaluation of a tailored, multi-component
intervention for implementation of evidence-based clinical practice guidelines in primary
care physical therapy: A non-randomized controlled trial. BMC Health Services
Research, 14(1), 1–15. https://doi.org/10.1186/1472-6963-14-105
Browning, M., Bilderbeck, A. C., Dias, R., Dourish, C. T., Kingslake, J., Deckert, J., Goodwin,
G. M., Gorwood, P., Guo, B., Harmer, C. J., Morriss, R., Reif, A., Ruhe, H. G., van
Schaik, A., Simon, J., Sola, V., Veltman, D. J., Elices, M., Lever, A. G.,...Dawson, G. R.
(2021). The clinical effectiveness of using a predictive algorithm to guide antidepressant
treatment in primary care (PReDicT): An open-label, randomised controlled trial.
Neuropsychopharmacology, 1–8. https://doi.org/10.1038/s41386-021-00981-z
Center for Disease Control and Prevention. (2017). Preventing suicide: A technical package of
policy, programs, and practices [PDF].
https://www.cdc.gov/violenceprevention/pdf/suicidetechnicalpackage.pdf

IMPLEMENTATION OF A SUICIDAL IDEATION TREATMENT

20

Center for Disease Control and Prevention. (2021, June 17). Emergency department visits for
suspected suicide attempts among persons aged 12-25 years before and during the
COVID-19 pandemic. Centers for Disease Control and Prevention.
https://www.cdc.gov/mmwr/volumes/70/wr/mm7024e1.htm
Conner, B. T. (2014, June 11). Differentiating research, evidence-based practice, and quality
improvement. American Nurse. https://www.myamericannurse.com/differentiatingresearch-evidence-based-practice-and-quality-improvement/
Curtin, S. (2020). State suicide rates among adolescents and young adults aged 10–24: United
States, 2000–2018. National Vital Statistics Reports, 69(11), 1–9.
https://www.cdc.gov/nchs/data/nvsr/nvsr69/nvsr-69-11-508.pdf
Dolan-Soto, D. R., Jonas, D. E., Reed, R. M., & Weil, A. B. (2020). Development of NAMASTE
(new anxiety management algorithm standardizing treatment experience) and
implementation in primary care. Psychiatric Services, 71(6), 627–630.
https://doi.org/10.1176/appi.ps.201900378
Graham, R. D., Rudd, M., & Bryan, C. J. (2011). Primary care providers’ views regarding
assessing and treating suicidal patients. Suicide and Life-Threatening Behavior, 41(6),
614–623. https://doi.org/10.1111/j.1943-278x.2011.00058.x
Hansoti, B., Jenson, A., Kironji, A. G., Katz, J., Levin, S., Rothman, R., Kelen, G. D., & Wallis,
L. A. (2017). SCREEN: A simple layperson administered screening algorithm in low
resource international settings significantly reduces waiting time for critically ill children
in primary healthcare clinics. PLOS ONE, 12(8), 1–13.
https://doi.org/10.1371/journal.pone.0183520

IMPLEMENTATION OF A SUICIDAL IDEATION TREATMENT

21

Haran, J. P., Goulding, M., Campion, M., Scully, G., Chandra, A., Goldberg, R., Day, A.,
McLendon, E., & Clark, M. A. (2020). Reduction of inappropriate antibiotic use and
improved outcomes by implementation of an algorithm-based clinical guideline for
nonpurulent skin and soft tissue infections. Annals of Emergency Medicine, 76(1), 56–66.
https://doi.org/10.1016/j.annemergmed.2019.12.012
Hauge, L., Stene-Larsen, K., Grimholt, T., Øien-Ødegaard, C., & Reneflot, A. (2018). Use of
primary health care services prior to suicide in the Norwegian population 2006–2015.
BMC Health Services Research, 18(1). https://doi.org/10.1186/s12913-018-3419-9
Hedegaard, H., Curtin, S., & Warner, M. (2021). Suicide mortality in the United States, 19992019 (No. 398) [National Center for Health Statistics Data Brief]. Center for Disease
Control. https://doi.org/10.15620/cdc:101761
Hogan, M. F., & Grumet, J. (2016). Suicide prevention: An emerging priority for health care.
Health Affairs, 35(6), 1084–1090. https://doi.org/10.1377/hlthaff.2015.1672
Hooper, L. M., Epstein, S. A., Weinfurt, K. P., DeCoster, J., Qu, L., & Hannah, N. J. (2012).
Predictors of primary care physicians’ self-reported intention to conduct suicide risk
assessments. The Journal of Behavioral Health Services & Research, 39(2), 103–115.
https://doi.org/10.1007/s11414-011-9268-5
Institute for Healthcare Improvement. (n.d.). Science of improvement: Testing changes.
Retrieved April 13, 2021, from
http://www.ihi.org/resources/Pages/HowtoImprove/ScienceofImprovementTestingChang
es.aspx
John, A., DelPozo-Banos, M., Gunnell, D., Dennis, M., Scourfield, J., Ford, D. V., Kapur, N., &
Lloyd, K. (2020). Contacts with primary and secondary healthcare prior to suicide: Case–

IMPLEMENTATION OF A SUICIDAL IDEATION TREATMENT

22

control whole-population-based study using person-level linked routine data in wales,
UK, 2000–2017. The British Journal of Psychiatry, 217(6), 717–724.
https://doi.org/10.1192/bjp.2020.137
Losey, S. (2021, April 5). Military deaths by suicide jumped 25% at end of 2020. Military.com.
https://www.military.com/daily-news/2021/04/05/military-deaths-suicide-jumped-25end-of-2020.html
Luke Air Force Base. (n.d.). 56th medical group. https://www.luke.af.mil/Units/56th-MedicalGroup/
Mendis, S., Johnston, S., Fan, W., Oladapo, O., Cameron, A., & Faramawi, M. F. (2009).
Cardiovascular risk management and its impact on hypertension control in primary care
in low-resource settings: A cluster-randomized trial. Bulletin of the World Health
Organization, 88(6), 412–419. https://doi.org/10.2471/blt.08.062364
Michail, M., Tait, L., & Churchill, D. (2017). General practitioners’ clinical expertise in
managing suicidal young people: Implications for continued education. Primary Health
Care Research & Development, 18(05), 419–428.
https://doi.org/10.1017/s1463423617000299
Miyar, M. E., Brown, M., Vivar, K. L., Jablon, K., Boscardin, C., Levy, M. L., Teng, J. M., &
Mathes, E. F. (2017). An atopic dermatitis management algorithm for primary care
providers and assessment of its usefulness as a clinical tool. Pediatric Dermatology,
34(4), 402–407. https://doi.org/10.1111/pde.13157
National Institute of Mental Health. (2021, January). Suicide.
https://www.nimh.nih.gov/health/statistics/suicide.shtml

IMPLEMENTATION OF A SUICIDAL IDEATION TREATMENT

23

Oravecz, R., & Moore, M. M. (2006). Recognition of suicide risk according to the characteristics
of the suicide process. Death Studies, 30(3), 269–279.
https://doi.org/10.1080/07481180500493492
Redmond, M. (2015). Uncertainty reduction theory. In English technical reports and white
papers (3rd ed.). Iowa State University.
https://doi.org/https://lib.dr.iastate.edu/engl_reports/3/
Richards, J. E., Whiteside, U., Ludman, E. J., Pabiniak, C., Kirlin, B., Hidalgo, R., & Simon, G.
(2019). Understanding why patients may not report suicidal ideation at a health care visit
prior to a suicide attempt: A qualitative study. Psychiatric Services, 70(1), 40–45.
https://doi.org/10.1176/appi.ps.201800342
Rodi, P. M., Roškar, S., & Marušič, A. (2009). Suicide victims’ last contact with the primary
care physician: Report from Slovenia. International Journal of Social Psychiatry, 56(3),
280–287. https://doi.org/10.1177/0020764009105330
Schaffer, A., Sinyor, M., Kurdyak, P., Vigod, S., Sareen, J., Reis, C., Green, D., Bolton, J.,
Rhodes, A., Grigoriadis, S., Cairney, J., & Cheung, A. (2016). Population-based analysis
of health care contacts among suicide decedents: Identifying opportunities for more
targeted suicide prevention strategies. World Psychiatry, 15(2), 135–145.
https://doi.org/10.1002/wps.20321
Schmitz, W. M., Allen, M. H., Feldman, B. N., Gutin, N. J., Jahn, D. R., Kleespies, P. M.,
Quinnett, P., & Simpson, S. (2012). Preventing suicide through improved training in
suicide risk assessment and care: An American Association of suicidology task force
report addressing serious gaps in U.S. mental health training. Suicide and Life-

IMPLEMENTATION OF A SUICIDAL IDEATION TREATMENT

24

Threatening Behavior, 42(3), 292–304. https://doi.org/10.1111/j.1943278x.2012.00090.x
Srivastava, A., Gailer, R., Tanwar, S., Trembling, P., Parkes, J., Rodger, A., Suri, D., Thorburn,
D., Sennett, K., Morgan, S., Tsochatzis, E. A., & Rosenberg, W. (2019). Prospective
evaluation of a primary care referral pathway for patients with non-alcoholic fatty liver
disease. Journal of Hepatology, 71(2), 371–378.
https://doi.org/10.1016/j.jhep.2019.03.033
Taylor, M. J., McNicholas, C., Nicolay, C., Darzi, A., Bell, D., & Reed, J. E. (2013). Systematic
review of the application of the plan–do–study–act method to improve quality in
healthcare. BMJ Quality & Safety, 23(4), 290–298. https://doi.org/10.1136/bmjqs-2013001862
The Department of Veteran Affairs & The Department of Defense. (2013, June). VA/DoD
clinical practice guideline for the assessment and management of patients at risk for
suicide [PDF]. The Department of Veteran Affairs.
https://www.aaaceus.com/courses/Articles/Ref_Guidelines_Risk_of_Suicide.pdf
The Department of Veteran Affairs & The Department of Defense. (2019). VA/DoD clinical
practice guidelines: The assessment and management of patients at risk for suicide
[PDF]. The Department of Veterans Affairs.
https://www.healthquality.va.gov/guidelines/MH/srb/VADoDSuicideRiskCPGProviderS
ummaryFinal5088212019.pdf
The United States Air Force. (2019). Suicide prevention program. U.S. Air Force Resilience.
Retrieved March 29, 2021, from https://www.resilience.af.mil/Suicide-PreventionProgram/

IMPLEMENTATION OF A SUICIDAL IDEATION TREATMENT

25

Williamson, M., Hogue, G., Cotter, J., Roberman, S., Neal, G., & Williamson, B. (2019). Suicide
screening: How to recognize and treat at-risk adults. The Journal of Family Practice,
68(9), 505–510. https://pubmed.ncbi.nlm.nih.gov/31725135/

IMPLEMENTATION OF A SUICIDAL IDEATION TREATMENT

26

Appendix A
Table 1
Quantitative Evaluation Table
Citation

Citation
Alexopoulos,
G. S., et al.
(2009)
Country
United States
Funding
Grant support
Cephalon &
Forest;
Pharmaceutical
company.
Bias
Pharmaceutical
associations
for multiple
authors.

Theoretical/
Conceptual
Framework
Treatment
algorithm based
on the Agency
for Health Care
Policy and
Research
Practice
Guidelines.

Design/
Method/
Purpose
Design
Randomized
control trial.
Purpose
Evaluate the
impact of a care
management
intervention on
suicidal
ideation and
depression in
older adults.

Sample/Setting

Variables

Measurement/
Instrumentation

Data
Analysis

Results/ Findings

N= 599
Demographics
Age 60+
PCP patients with
major depression.
Female= 72%
Minority= 32%
Majority single,
lived alone
Setting
20 PCP offices;
urban, suburban &
rural.
Pairs of
socioeconomically
equivocal offices
were formed &
then control
offices were
randomly
assigned for each
pair
Exclusion
Age <60
Attrition
At 24 months
43% of
intervention and
37% of control
failed to complete
assessment

IV1
PROSPECT
program

Structured
interview
HDRS
MMSE
SSI

Omnibus
statistics test,
SAS PROC
MIXED,
NLMIXED,
& GLIMMIX

Reported SI:
treatment group >
than control
4 month: 12.8% vs
3.0% P=0.02
24 month: 18.3% vs
8.3% P=0.12

DV1
Reported SI
DV2
Course of
depression
(Severity,
Treatment
response,
Remission)
Definition
Treatment=
decreased
HDRS score
≥50%
RemissionHDRS <7

Severity: Greater in
baseline scores
intervention group.
Response: Higher
response
intervention group.
x2(5)=17.3 P<0.004
Remission: Overall
increased in
treatment group only
until 8 months. Then
equal for both
groups. Major
depression remission
rates were higher in
treatment group at
24 months.

Level of Evidence;
Application to practice/
Generalization
Level of Evidence- 2
RCT
Strengths
Long study time.
Weakness
Only evaluated older
adults. Decreases
generalizability for use
in all age groups.
Blinding was not
possible due to the lack
of ancillary staff and
algorithm use in the
control group.
Feasibility
The increased cost
associated with
incorporating care
managers may reduce
feasibility of
implementation.
Not generalizable. QI
project specific for site
where it was
implemented.

Key: DV-dependent variable; EBP-Evidence Based Practice IV-independent variable; HDRS-Hamilton Depression Rating Scale; MMSE-Mini
Mental State Examination N- number of studies; n-number of participants; NA-Not applicable PCP-primary care providers; QI-Quality Improvement
SI- suicidal ideation; SSI-Scale for Suicidal Ideation; vs-versus;

IMPLEMENTATION OF A SUICIDAL IDEATION TREATMENT
Citation
Citation
Amin, R., &
Thomas, M.
(2020)
Country
United States
Funding
Not reported
Bias
Military
setting. Author
worked in
clinic of study.

Citation
Aminsharifi,
A., et al.
(2018)
Country
United States
Funding
Duke Cancer
Institute
federal grant
Bias
None

Theoretical/
Conceptual
Framework
QI project.
Curriculum
based on Plan,
Do, Study, Act
framework.

Design/
Method/
Purpose
Design
One group
pretest and
posttest
intervention
survey.
6 month follow
up survey.
Purpose
Evaluate PCP
attitudes,
knowledge, and
skill in treating
mental health
patients after
intervention.

QI project.
Evidence based
treatment
algorithm
developed.

Design
One group
posttest survey
& chart review
Purpose
Assess practice
changes and
provider
attitudes after
algorithm
implementation.

27

Sample/Setting

Variables

Measurement/
Instrumentation

Data
Analysis

Results/ Findings

N= 35
Demographics
Primary Care
Physicians-4
Physician
assistants- 31

IV1Depression
and anxiety
management
training and
decisional tool.

Likert scale
survey.

Performed on
IBM SPSS;

Mean years in
practice 3.63
Setting
Military PCP
Exclusion
NA
Attrition
Immediate-0%
6-month -30 %

DV1- Provider
confidence &
perceived
knowledge

Post-intervention
DV1: Confidence
t(35)= −3.509, P <
0.001, d = 1.06
Perceived
knowledge: t(37) =
−3.554, P < 0.001, d
= 1.08.
Follow up:
DV2: Intervention
led to practice
change- t(9)=−2.714,
P < 0.02, d = 0.58.
DV1: Subjective
reports decision tool
very helpful.

N=106
Demographics
PCP
Setting
All primary care
offices within a
large healthcare
organization.
Patient chart
review
Men
Age-40-74
Exclusion
NA
Attrition
NA

IV1- Prostate
cancer
screening
algorithm

Independent
T-test,
Paired T-test,
Cohen d.

DV2
Psychotropic
prescribing
practices

DV1
Reported
provider
confidence
DV2
Number of
prostate
screenings

Likert scale
survey
Chart review

Raw data
reported

DV1: 79% of PCP
reported feeling very
confident in
screening patients.
DV2: Prostate
screening rates
increased by 9%

Level of Evidence;
Application to practice/
Generalization
Level of Evidence 4
Cohort study
Strengths
Follow up survey
provides data regarding
long term
improvements.
Appropriate statistical
analysis performed.
Weakness
Study only conducted in
one clinic. Low level of
evidence. Low follow up
response rate.
Feasibility
Not generalizable. QI
project specific for site
where it was
implemented.
Level of Evidence 4
Cohort study
Strengths
Evaluation of provider
perception and statistical
information on treatment
changes.
Weakness
No pre/post intervention
survey. No information
provided on statistical
data collection methods.
Feasibility
Not generalizable. QI
project specific for site
where it was
implemented.

Key: DV-dependent variable; EBP-Evidence Based Practice IV-independent variable; HDRS-Hamilton Depression Rating Scale; MMSE-Mini
Mental State Examination N- number of studies; n-number of participants; NA-Not applicable PCP-primary care providers; QI-Quality Improvement
SI- suicidal ideation; SSI-Scale for Suicidal Ideation; vs-versus;

IMPLEMENTATION OF A SUICIDAL IDEATION TREATMENT
Citation
Citation
Bernhardsson
et al., (2014)
Country
Sweden
Funding
Doctoral grant
from
Linköping
University
Bias None

Theoretical/
Conceptual
Framework
QI project
5-step model
for
implementation

28

Design/
Method/
Purpose
Design
Nonrandomized
control trial.
Purpose

Sample/Setting

Variables

Measurement/
Instrumentation

Data
Analysis

Results/ Findings

N= 425
Demographics
Primary care
physical therapists

Self-report
questionnaire.

Power 80%

Evaluate the
effect of a
guideline to
improve
practice.

Setting
60 primary care
clinics

IV1 Treatment
guideline
DV1
Provider
knowledge
DV2
Attitude about
guidelines

DV1
Awareness:
Intervention (59%)
Control group
(44%) reported (p =
0.030),
Finding guidelines
(40% vs. 16%; p <
0.001),
Access to guidelines
(26% vs. 7%; p
<0.001).
DV2
Intervention group
considered guideline
helpful p=0.018

EpiServer survey
software.

Significance
level 5%
Pearson X2
z-test

Exclusion
Employed by
county council

SPSS

Attrition 40%

Citation
Browning, M.,
et al. (2021)
Country
United
Kingdom,
Spain,
Germany,
France,
Netherlands
Funding
European
Union’s
Horizon 2020
research
Bias None

QI project
Affective
processing bias

Design
RCT
Purpose
Improve
medication
management of
depressed
patients

N= 913
Demographics
White 90%
Female 62%
Mean age-40
range 18-70
Setting
Primary care
clinics in 5
European
healthcare
systems
Attrition 50% by
12 months

IV1 PReDicT
DV1
Depression
remission
DV2
Anxiety
remission
Definition:
PReDicTPredictive
algorithm for
antidepressant
treatment
Remission50% reduction
in symptoms

Quick Inventory
of Depressive
Symptoms
Montgomery–
Åsberg
Depression
Rating Scale
Generalized
Anxiety Disorder
Assessment

Alpha twotailed 0.05,
Power 80%establish
sample size.
Pre-published
protocol,
statistical
analysis.
Multilevel
logistic
regression,
odds ratio,
95%
confidence
interval

DV1: PReDicT
(55.9%) & control
(51.8%) arms did not
differ significantly
(odds ratio: 1.18
(95% CI: 0.89–1.56),
P = 0.25)
DV2: PReDicT
Mean change −9.70
95%CI (−10.79,
−8.61) Control:
Mean change −7.48
95%CI (−8.60,
−6.36) 2.22 (0.74,
3.70) p=0.004

Level of Evidence;
Application to practice/
Generalization
Level of Evidence 3Nonrandomized
controlled study
Strengths
Multiple clinics.
Weakness
High attrition rate. Not
randomized.
Feasibility Feasible to
reproduce
Not generalizable. QI
project specific for site
where it was
implemented.

Level of Evidence 2
RCT
Strengths
Implemented in multiple
countries.
Weakness
Algorithm sensitivity
57%. Adherence to the
protocol prescribing
recommendation was
inconsistent in treatment
group.
High attrition.
Feasibility: Feasible to
reproduce
Not generalizable. QI
project specific for site
where it was
implemented.

Key: DV-dependent variable; EBP-Evidence Based Practice IV-independent variable; HDRS-Hamilton Depression Rating Scale; MMSE-Mini
Mental State Examination N- number of studies; n-number of participants; NA-Not applicable PCP-primary care providers; QI-Quality Improvement
SI- suicidal ideation; SSI-Scale for Suicidal Ideation; vs-versus;

IMPLEMENTATION OF A SUICIDAL IDEATION TREATMENT
Citation
Citation
Dolan-Soto,
D.R., et al.
(2020)
Country
United States
Funding
Self-Funded
Bias
None

Citation
Hansoti, B., et
al. (2017)
Country
South Africa
Funding
Expanded
Public Works
Program
Bias
Not reported

Theoretical/
Conceptual
Framework
QI project
utilizing a
newly
developed
program
(treatment
algorithm,
medication
resource, and
care-based
learning
module).
Modeled after
IMPACT
program for
depression
treatment.
QI project
Integrated
Management of
Childhood
Illness clinical
case
management
guideline.

Design/
Method/
Purpose
Design
Post
intervention
survey.
PurposeImprove
anxiety disorder
recognition and
treatment.

Design
Mixed methods;
Direct
observation, pre
& post
implementation
chart-review
Purpose
Rapidly identify
critically ill
children and
expedite their
care.

29

Sample/Setting

Variables

Measurement/
Instrumentation

Data
Analysis

Results/ Findings

Demographics
N=40 Residents
N=18 Attending
physicians
Setting
Large outpatient
Primary care
clinic in North
Carolina; 100
providers seeing
1,200 patients

IV1
NAMASTE
protocol
DV1
Provider
confidence
DV2
Anxiety
diagnosis and
appropriate
treatment

Post intervention
anonymous
survey.

Absolute
values.

Reported as
% of providers that
agreed with
statements.
DV1: Improved
ability to prescribe:
Residents: 95%
Attendings 100%
DV2: Improved
diagnosis:
Residents 59%
Attendings 72%
Improved ability to
treat:
Residents 73%
Attendings 77%

Observation &
chart review
N= 3383
Demographics:
Pediatric patients
Pre/Post chart
review
N=827
Demographics:
Charts had
SCREEN & IMCI
Setting
Primary
healthcare
pediatric clinic

IV1 SCREEN
treatment
algorithm

Custom android
app to randomly
assign patient
numbers.

STATA v.12Coxregression

DV1: Screened
within 5 minutesmedian 84.1%, IQR
66.3%-90.3%,
ANOVA R=0.2859
p<0.0001
Nurse within 10
minutes- Median
83.5%, IQR 38.8%100%, ANOVA R=
0.3936, p<0.0001
Median time
decreased from
100.3 minutes to 4.9
minutes P <.001
DV2: Tool
sensitivity- 94.2%
specificity 88.1%

DV1
Time until
critically ill
patients see a
provider
DV2
Accurately
identified
critically ill
patients

Survey results
can be assumed
to be reliable and
valid.
Not enough
information is
reported to
assess reliability.

Microsoft Excel
spreadsheet to
track times.

ANOVA

Level of Evidence;
Application to practice/
Generalization
Level of Evidence: 3Nonrandomized
controlled study
Strengths
Research was performed
in a large clinic.
Weakness
No control and study
group. No pre
implementation
information.
Feasibility
Not generalizable. QI
project specific for site
where it was
implemented.
Level of Evidence 4
cohort study
Strengths: Large study.
Calculated number
needed to observe for
validity.
Weakness: No
randomization.
Implementation
inconsistency between
offices.
Feasibility
Not generalizable. QI
project specific for site
where it was
implemented.

Key: DV-dependent variable; EBP-Evidence Based Practice IV-independent variable; HDRS-Hamilton Depression Rating Scale; MMSE-Mini
Mental State Examination N- number of studies; n-number of participants; NA-Not applicable PCP-primary care providers; QI-Quality Improvement
SI- suicidal ideation; SSI-Scale for Suicidal Ideation; vs-versus;

IMPLEMENTATION OF A SUICIDAL IDEATION TREATMENT
Citation
Citation
Mendis et al.,
(2009)
Country
United States,
China &
Nigeria
Funding
Not reported
Bias Not
reported

Citation
Miyar, M. E.,
et al. (2017)
Country
United States
Funding
Conducted by
Universities.
No funding
information
provided.
Bias None

Theoretical/
Conceptual
Framework
EBP.
World Health
Organization
treatment
guidelines.

QI project
Best practice
guidelines and
American
Academy of
Dermatology
guidelines.

Design/
Method/
Purpose
Design
RCT
Purpose
Assess
effectiveness of
coronary
vascular disease
algorithm to
reduce blood
pressure.

Design
Pre and post
intervention
surveys
Purpose
Improve PCP
knowledge on
Atopic
Dermatitis
management.

30

Sample/Setting

Variables

Measurement/
Instrumentation

Data
Analysis

Results/ Findings

N= 2397
Demographics
Age: 30-70 with
blood pressure
140-179
Setting
10 pairs primary
care offices in
China & Nigeria
Exclusion
Comorbid
condition that may
cause secondary
hypertension
Attrition 25%

IV1
Cardiovascular
risk
assessment &
management
algorithm
DV1
Blood pressure
DV2
Smoking
cessation,
BMI, eating
habits

Registry,
inclusion visit,
follow up visit,
and exit
interview forms.

X2 α, 0.05:
power,
interclass
correlations

DV1- Median
increase in target
blood pressure of
16.2% (interquartile
range, IQR: 10.3–
32.2) and 6.0%
(IQR: 1.5–17.5)

N= 78
Demographics
PCP residents and
attending
physicians
Setting
PCP office
residency
programs
Attrition 30%

IV1 Atopic
dermatitis
treatment
algorithm
DV1
Provider
knowledge
DV2
Perception of
algorithm
usefulness

Multichoice
knowledge
assessment and
survey

Visual check of
quality.

unvaried
comparisont-test
DichotomousFisher exact
test

DV2- Not significant

Level of Evidence;
Application to practice/
Generalization
Level of Evidence 2
RCT
Strengths
Large study.
Weakness
Patients lost to follow up
were the most ill. Likely
affecting data.
Feasibility
Generalizability limited
by setting.

ANOVA

Analysis of
covariance
ANCOVA

DV1- 1.19 points
higher on average on
the posttest (b = 1.19
[95% confidence
interval 0.07, 2.32],
p = 0.04)
DV2Intervention group
participants- 89%
would use algorithm
for AD

Level of Evidence 2
RTC
Strengths-Conducted in
multiple states. High
level of research.
Weakness Small sample
size. Possible crosseducation. Control and
intervention worked in
the same office.
Feasibility Simplicity of
resources increases
feasibility.
Not generalizable. QI
project specific for site
where it was
implemented.

Key: DV-dependent variable; EBP-Evidence Based Practice IV-independent variable; HDRS-Hamilton Depression Rating Scale; MMSE-Mini
Mental State Examination N- number of studies; n-number of participants; NA-Not applicable PCP-primary care providers; QI-Quality Improvement
SI- suicidal ideation; SSI-Scale for Suicidal Ideation; vs-versus;

IMPLEMENTATION OF A SUICIDAL IDEATION TREATMENT
Citation
Citation
(Srivastava et
al., 2019)
Country
United
Kingdom
Funding
National
Institute of
Health
Research

Theoretical/
Conceptual
Framework
EBP
Care pathway

Design/
Method/
Purpose
Design
Pre and post
implementation
comparison
Purpose
Improve
appropriate
diagnosis of
non-alcoholic
fatty liver
disease &
decrease
inappropriate
referrals

31

Sample/Setting

Variables

Measurement/
Instrumentation

Data
Analysis

Results/ Findings

N= 3,012
Demographics
Adult primary
care patients with
new nonalcoholic
fatty liver disease
diagnosis
Setting
Primary care
clinics

IV1
Nonalcoholic
fatty liver
disease
algorithm
DV1
Inappropriate
referrals
DV2
Appropriate
referrals

Electronic
patient records
identified with
reports by
diagnosis and
use of algorithm
then reviewed
individually by
research team.

SPSS, odds
ratio, 95%
Confidence
intervals, chisquare tests

DV1 Detected 5
times more cases of
advanced fibrosis
and cirrhosis (odds
ratio [OR] 5.18; 95%
CI 2.97–9.04;
p <0.0001)
DV2 improved
clinical judgementreduced unnecessary
referrals to
secondary care by
81% (OR 0.193;
95% CI 0.111–
0.337;
p <0.0001)

Level of Evidence;
Application to practice/
Generalization
Level of Evidence 4
Cohort study
Strengths
Large study.
Implemented in realworld setting
Weakness
No randomization or
blinding
Feasibility
Feasible to implement.
Generalizable in Primary
care clinics.

Key: DV-dependent variable; EBP-Evidence Based Practice IV-independent variable; HDRS-Hamilton Depression Rating Scale; MMSE-Mini
Mental State Examination N- number of studies; n-number of participants; NA-Not applicable PCP-primary care providers; QI-Quality Improvement
SI- suicidal ideation; SSI-Scale for Suicidal Ideation; vs-versus;

IMPLEMENTATION OF A SUICIDAL IDEATION TREATMENT

32

Appendix B
Table 2
Synthesis Table

Author
Year
LOE
Framework
QI
EBP
Setting
Population
Patients
Providers
Data type
Subjective symptoms
Physiologic medical
Data collection method
Survey
Interview
Screening score
Chart review
Intervention
Algorithm/
Decisional tool
Med guide
Patient Outcomes
Appropriate Treatment
Appropriate Referrals
Symptoms
Outcomes
Provider Outcomes
Knowledge
Confidence
Treatment Practice

Alexopoulous Amin
2009
2020
2-RCT
4-Cohort

Aminsharifi
2018
4-Cohort

Bernhardsson Browning Dolan-Soto Hansoti
2014
2021
2020
2017
3-NRCT
2-RCT
3-NRCT
4-Cohort

X

X

X

X

PCP office

PCP office PCP office

X
X

X

X

X
Pre/post

X

X

X

PCP office

X
X

X

X

X

Post

Post

X
X

X

X

X

X
Post

X

X

X

↑

↑

X

X

Miyar
2017
2-RCT
X
PCP office

Srivastava
2019
4-Cohort
X
PCP office
X

X
X

X

Pre/Post

X

X

X

X

X

X

X

X

X
X

X

X
↑

↑

X

X

↓
↑
↑
↑
↑

X

X

↑

↓
↑

↑

X

X
PCP office PCP office PCP office PCP office
X

X
↑

X

Mendis
2009
2-RCT

↑
↑

↑
↑
↑

↑
↑
↑ NS
↑
↑

Key: EBP-Evidence Based Practice; LOE- Level of Evidence; NRCT- Nonrandomized control trial; NS- Not Significant QI-Quality Improvement;
RCT- Randomized control trial

IMPLEMENTATION OF A SUICIDAL IDEATION TREATMENT

33