This document serves to update National Institute of Mental Health (NIMH) guidance on suicide intervention research 1 . It is intended to support the development of NIMH research grant applications in suicide research, including those related to clinical course, risk and detection, and interventions and implementation, as well as to support research conduct that is safe, ethical and feasible. As part of this effort, NIMH is encouraging mental health researchers more broadly to ask questions about suicide and suicide risk as part of their study assessments in research that may not include intentional self-injury as a primary outcome. This guidance aims to be of use to these investigators as well. It is not intended to provide enforceable guidelines for researchers, or to establish policy for any agency on human subject research protections.
In 2017, suicide was the 10th leading cause of death in the United States, with more than 47,000 deaths (CDC ) 2 . In addition to the burden of mortality, 10.6 million adults aged 18 or older reported thinking seriously about trying to kill themselves in 2017, including 3.2 million who reported making suicide plans and 1.4 million who made a nonfatal suicide attempt 3 . Over the past several decades, suicide prevention efforts have yet to significantly reduce the rates of suicide and suicide attempt in the United States 4 .
To address this public health crisis, NIMH helped develop the Prioritized Research Agenda for Suicide Prevention , requested by the National Action Alliance for Suicide Prevention (NAASP), a public-private partnership among federal agencies, state governments, private sector companies, and national suicide prevention advisory and advocacy groups. The Agenda identifies research gaps, such as limitations in understanding of the etiology and course of suicidal ideation and behavior, the need for new interventions specifically targeting suicide, better matching of existing treatments to individual needs, and implementation of effective suicide prevention practices. The Action Alliance’s Zero Suicide effort has shaped suicide research in this country by identifying the implementation steps needed to shift the culture of health care to include more practices that actively support suicide prevention 5 . NIMH has solicited applications to increase the evidence base for suicide prevention practices in health care systems (e.g., RFA-MH-16-800 ). NIMH is also participating in the Helping to End Addiction Long-term SM Initiative, or NIH HEAL Initiative SM , funding studies seeking to identify effective collaborative care models that provide integrated treatment of opioid use disorders and mental health conditions (RFA-MH-19-525 ). A primary research question for these projects is whether collaborative care reduces risk for premature mortality, including suicide via intentional overdose or other means. Both of these efforts are consistent with the NIMH Strategic Plan for Research, Objective 4, improving public health through more practice-ready suicide prevention interventions.
Developing additional novel suicide prevention and treatment approaches, testing best practices in pragmatic trials, and asking about suicide in mental health research more generally, present a range of safety and ethical questions. Some of these questions are familiar to the field, for instance:
Below are cross-cutting ethical and participant safety questions that researchers should consider when including suicidal or potentially suicidal participants in their research. In addition, new ethics and safety questions are arising from scientific, methodological, and technological developments in the field, such as the use of emerging technologies in clinical research and the increased adoption of pragmatic effectiveness study designs. These will be discussed as emerging questions.
Notably, in October 2016, NIMH was part of a group of NIH Institutes issuing a Notice of Interest in High Priority Research in Bioethical, Legal, and Societal Implications of Biomedical Research (NOT-LM-17-001 ). The high priority research areas include numerous topics addressed below: new and emerging technologies, data sharing, learning health care systems/pragmatic trials, informed consent, and regulatory and oversight issues.
Although not specific to research on suicide, the Office of Human Research Protections (OHRP) announced revisions to the Federal Policy for the Protection of Human Subjects (the Revised Common Rule) in January 2017 with the goal of better protecting “human subjects involved in research, while facilitating valuable research and reducing burden, delay, and ambiguity for investigators. These revisions are an effort to modernize, simplify, and enhance the current system of oversight.” One area of the revision relevant to suicide research relates to new exemptions for the secondary use of identifiable private information and biospecimens, and the de-identified use of previously identifiable private information and biospecimens. Some suicide studies that would previously have been considered human subject research may now be eligible for limited IRB review or exemption from IRB review. For example, under the revised rule, private information and biospecimens could qualify for exemption even if they are not collected prior to the start of a study, but will be collected for clinical use in the future. In addition, a new provision allows secondary use of identifiable private information to be exempt from the regulation when this data is regulated under HIPAA as part of health care operations, research, and public health, such as use of medical record data authorized for research intended to impact public health.
Related to these new exemptions on the use of identifiable data is Common Rule language on Broad Consent, (i.e., asking whether or not an individual is willing to allow their private identifiable information or biospecimens to be available to a researcher for some defined future use). Requesting Broad Consent involves contingency planning because if an individual declines, their identifiable data can no longer be considered for the requested future use, although their de-identified data would still be available. To determine whether a specific study is exempt from this regulation, and whether to consider using Broad Consent, consult with the institutional Human Research Protection Program overseeing the research.
The NIMH is strongly encouraging data sharing and the use of common data elements (NOT-MH-19-033 ; NOT-MH-15-009 ). These efforts are vital in the field of suicide research because suicide attempt and suicide death are low base rate events, thus the value of individual studies is significantly enhanced when researchers collect and share data that can be harmonized with other suicide-related research studies. One aspect of data harmonization involves using assessment instruments being used by others in the field, following the guidance provided in NIMH clinical trial FOAs (e.g., PhenX Toolkit ). Another aspect of data collection to consider with regard to harmonization relates to data extracted from setting-specific sources, such as electronic health records (EHR), in contrast to data collected using standardized assessment instruments. Researchers may want to think through and collaborate around which elements in the medical record will best synchronize with data from other studies. Efforts at cross-walking data elements during the development of a study protocol will promote data harmonization, and in turn improve the likelihood of data sharing. As more studies use EHR data sources, policy and practical questions related to harmonizing data across EHR systems will become more salient.
For those clinical trials developing and evaluating drugs developed for psychiatric indications, investigators are referred to the Food and Drug Administration’s (FDA) draft Guidance for Industry, Suicidal Ideation and Behavior: Prospective Assessment of Occurrence in Clinical Trials . In 2012, the FDA issued this draft guidance on the assessment of suicidal ideation and behavior in psychiatric and non-psychiatric drug trials using compounds under development. This guidance provides an informative framework for investigational new drug (IND) studies conducting these prospective assessments, and also encourages common safety data be collected across studies to allow for meta-analysis of these data. Language in the guidance can help investigators determine whether these expectations apply to their research, as well as recommendations on the timing and content of the assessments.
More recently, a June 2018 FDA draft Guidance for Industry, Major Depressive Disorder: Developing Drugs for Treatment addresses the inclusion of individuals with histories of suicidal ideation and behavior in studies working on pharmacological treatments for Major Depressive Disorder. The guidance specifies that, “Patients with a history of suicidal ideation and behavior need not be systematically excluded from trials.” The FDA’s statement is part of the federal effort to encourage researchers who are not involved in suicide prevention specifically to include participants at elevated risk in their studies.
With greater awareness of suicide as a significant public health problem, some states are passing laws requiring health care professionals to complete suicide prevention trainings, and others have policies recommending training 6 . Although these laws may not be specific to researchers, they serve as reminders of the need to be knowledgeable about state and local laws and policies, in addition to federal legislation and guidance. The informed consent discussion below notes various consent topics where consultation with someone expert in state and local mental health laws is recommended. In the field of suicide research, local knowledge is also useful in the acquisition of official death records or certificates for confirmation of a death and cause of death data. There may be options for how to obtain these records in a given state or locality (e.g., state departments of health, state medical examiner’s or coroner’s office) that vary in the process, timeline, and cost of obtaining official death data. For instance, investigators need to be knowledgeable about what key identifying data they need from study participants to reliably and efficiently access vital records from state or federal sources, as requirements for what identifying information is needed to request records in specific localities may vary.
Cross-cutting considerations are discussed below, with the goal of identifying issues that researchers may find useful to address when developing a research project involving participants at heightened risk for suicide.
Is the study setting and population appropriate for the aims of the suicide relevant research project with regard to safety, ethical approaches, and the knowledge to be gained?
The safe and ethical design of research studies enrolling participants at heightened risk of suicide (and those asking questions about suicidal ideation and/or intentional self-injury in non-suicide focused research) raises unique considerations for researchers. Methodological questions need to be addressed, as well as practical considerations about the physical environment and clinical setting where study procedures, including risk monitoring and safety planning, will be taking place. The physical environment where study activities are completed represents an important consideration for studies including individuals at risk for suicide 7 . Study visits can involve participants spending time in spaces that include items which could be used as means for self-harm (e.g., wires, needles, heights, glass, medications, poisons). Thinking through these physical space considerations during the planning phases of a project allows researchers to develop procedures to protect participants. Increasingly, study activities are being conducted remotely, with participants completing intervention and follow up procedures via electronic devices. Safety and ethical considerations that arise with study designs that involve remote participation will be discussed below in relation to safety monitoring and reporting, responding to suicidal crises and clinical worsening, and the use of emerging technologies.
Relatedly, researchers conducting studies in settings where participants are receiving clinical care (e.g., inpatient units, emergency rooms, outpatient clinics) will benefit from thinking through how the research staff and clinical providers will communicate with each other about patient needs and clinical status, and what will help study participants have a clearer understanding of the difference between their clinical care and their research participation. For example, studies that include assessments of suicide risk with patients in an emergency room or outpatient context will want to determine in advance, and in collaboration with the clinical providers in the setting, how a positive risk screen will be handled in order to assure that patients receive appropriate and timely care regardless of whether they participate in the research study. As more health care settings utilize EHR systems, safety planning templates and reminders will likely be more standardized within a given health system 8 . Researchers working in those settings will need to collaborate closely with health care system partners to develop their safety protocols in the context of existing systems, which will serve to facilitate communication and establish clear safety protocol workflows between research and clinical care teams.
Researchers can take advantage of existing tools to assist them in pro-actively thinking through potential safety, ethical, and care delivery issues that could arise over the course of study operations 9 . The Institute for Healthcare Improvement has developed a program that can be adopted for this purpose, the Failure Modes and Effects Analysis Tool. 10
Finally, ethically-informed decisions regarding the design and methods for a study involve deliberate consideration of the study population, and the ethical questions that arise when conducting mental health research with a given community, or with diverse communities with respect to culture and identity 11 . There are increasing efforts underway to include the perspectives of individuals who have direct experience with suicide in the development of research questions on suicide risk and effective prevention, and related study designs 12 , 13 . The Now Matters Now website provides an example of a suicide prevention resource developed by individuals with lived experience who are researchers, clinicians and consumers 14 ; the researchers who are part of Now Matters Now are studying the intervention and will provide data on its effectiveness to the larger community 14 , 15 .
There have also been efforts to address the specific ethical challenges of conducting suicide research in particular highly impacted communities. The American Indian/Alaska Native (AI/AN) Task Force of the NAASP convened an expert meeting in 2013 to address how to advance suicide prevention research with rural AI/AN populations. The 2015 report from that meeting provides useful ideas for researchers interested in conducting ethical suicide prevention research with native communities 16 . Consistent with this effort, researchers have recently tested a culturally tailored research ethics training for AI/AN communities to assess whether use of this curriculum with potential AI/AN research participants might increase participation in research affecting their communities 17 . The results suggest that attending to research participants’ culture and identity in relation to their roles as human subjects contributes to ethical and inclusive research. These examples of culturally-informed human subject protection initiatives and collaborative research approaches with individuals with lived experience provide models for approaching suicide research in a way that keeps affected communities’ lives in the forefront.
For suicide prevention trials, does the design allow for a comparator that is both informative and ethical?
In the arena of methodological issues, efficacy and/or effectiveness studies of novel suicide prevention interventions have the challenge of determining an appropriate comparator arm when conducting trials, as there may be no established effective standard of care within that setting for reducing suicidal ideation and behavior in the population specified for study, or appropriate for the research intervention and study design. In these situations, ethical questions arise, such as risk to suicidal individuals of being assigned to the “placebo arm” versus the active intervention, or risks raised by engaging in an untested intervention. These dilemmas can often be addressed by providing risk monitoring and safety planning throughout the study. Risk monitoring and safety planning may significantly enhance the care typically available. In these cases, investigators will be concerned with how the enhancement of usual care will impact their ability to determine if the new treatment is a significant improvement over true, unenhanced standard care. The enhanced treatment as usual (TAU) approach brings up another ethical dilemma: as more safety measures are put in place for the comparator TAU arm, studies then need to enroll more individuals to be appropriately powered to find a statistically and clinically significant effect, leading to more participants being enrolled and exposed to risks that may come with study involvement.
Different questions arise for studies that look at suicide as an outcome but are not addressing suicide reduction as a primary study aim, i.e., a treatment study of individuals with mood disorders. In these cases, established psychosocial and/or pharmacological treatments can be used as the active comparison group. Monitoring and risk management plans for suicidal ideation and suicide attempt will need to be developed that are appropriate for the study design and for the population, depending on the study inclusion criteria. Studies attempting to maximize external validity may elect to have broader inclusion criteria which can result in including individuals in the study who are at increased suicide risk. These design decisions have implications for safety monitoring and clinical risk management plans as noted above, as well as adverse event reporting expectations (e.g., considerations of what will be expected adverse events, and the rates of those events). As an example, studies that include individuals with substance abuse disorders might consider a psychiatric hospitalization for detox an expected serious adverse event, while this type of hospitalization might be considered unexpected with a study population that excluded current substance abuse.
What elements should be considered for inclusion as part of the informed consent process?
Informed consent in suicide-related research raises a number of issues, some of which are unique to suicide research and some which have heightened importance because of the risk issues presented in these studies. A primary issue arises in studies consenting individuals who may be in the midst of, or coming out of, a mental health crisis. In addition to considering if there are factors that may alter a person’s capacity to consent for themselves or their child, and for a child (e.g., a person under the age of 18) to assent to participation, the study team needs to communicate that involvement in research does not provide participants with “protection” against future suicidal behavior 18 . A different set of issues arises for suicide prevention studies that seek waivers or alterations of consent; see references in discussion of pragmatic trials below.
The language on confidentiality has particular relevance in studies with a higher likelihood that confidentiality would need to be limited due to concerns about imminent risk of suicide. Similarly, informing individuals, and families when relevant, of the clinical management plans in place in case of imminent suicide risk is an important aspect of safe and ethical research. A related discussion with study participants (and, in some case, family members) is needed at the outset of a study: addressing potential role confusion between a participant’s clinical care providers and study staff 19 . In situations when both care providers and study staff may be assessing for suicide risk or providing related care, participants will need guidance on who to reach out to and in what circumstance, and what information stemming from the research study is shared with their clinical providers.
Informed consent documents often include language regarding Health Insurance Portability and Accountability Act (HIPAA) Privacy Rule protections. Federal guidance on the limits to HIPAA in situations involving concerns about imminent suicide risk should be reviewed (see HIPAA Privacy Rule and Sharing Information Related to Mental Health ). State laws may have specific provisions related to limits of HIPAA when there is concern about harm; consulting with the legal counsel of state professional associations can provide helpful state-specific information regarding how to navigate privacy protections and their limits when concerns about self-harm arise.
Another example of state-specific limits to confidentiality relevant to suicide-related research involves Extreme Risk Protection Orders, also known as ‘red flag laws’ 20 . These laws allow concerned citizens to contact law enforcement to initiate an evaluation to determine whether time-limited risk-based gun removal is indicated for a specified individual exhibiting dangerous, or potentially dangerous behaviors, who has firearm access. As such, consultation with an attorney knowledgeable about state mental health law would be necessary to determine if there is a ‘red flag law’ in place in the state (or states) where the research is taking place. Based on the specifics of the state law, this consultation can provide guidance on what should be included in an informed consent document and assist study teams in thinking through privacy and confidentiality issues that would inform safe practice procedures in their specific research context should a situation involving gun access and suicide risk arise.
Also related to confidentiality considerations, effective October 2017, all NIH-funded research studies which collect identifiable, sensitive information are automatically issued a Certificate of Confidentiality by the Department of Health and Human Services. Other confidentiality considerations include whether there are options for stronger protections to take into account for the specific study population. For example, those doing research with prisoners (and others with arrest histories) can apply for a Privacy Certificate from the National Institute of Justice that provides a higher level of confidentiality protections to study participants.
In a discussion about informed consent with youth at elevated risk for suicide, King and Kramer (2008), describe a number of issues to consider, including the need for study teams to be knowledgeable about state laws on emancipated minors, mature minors, and other special circumstances. State law should also be consulted regarding the question of whether it is allowable to consent individuals who are involuntarily committed in a psychiatric facility 21 .
Another set of issues needing attention during the informed consent process relates to the time period for contact after active study participation. As noted above, suicide attempt and suicide death are low-base rate events; extending the data collection period is one way to increase the chances of capturing a higher event prevalence. Asking potential participants if they are willing to be contacted after the initial study is completed to participate in either follow-up or additional related research is crucial for progress in this field. It is also especially important to include appropriate language on data sharing to facilitate learning across studies. The data sharing language should be explicit about the degree of data sharing that could take place. A link to sample data sharing language for informed consent forms is available on the NIMH Data Archive (NDA) website.
Furthermore, research with suicide death as an outcome can include ascertaining participant-level information from state or national death records, e.g., the National Death Index 22 . Informed consent language permitting the study to access personally identifiable information (PII) from specified databases can facilitate vital statistics searches.
What level of data and safety monitoring is appropriate for the study?
Monitoring of clinical research refers to a study’s ongoing and systematic review of data integrity, protocol adherence, and participant safety. As such, monitoring includes: 1) the clinical/safety monitoring of individual participants; 2) the monitoring of safety data across treatment arms and/or across sites; 3) monitoring of study documents and study data to ensure that research is being conducted according to the standards of good clinical practice (GCP ) and in accordance with the approved study protocol; and 4) review of study data to assure data accuracy and consistency from collection through storage, retrieval, analysis, and reporting. With the risk and ethical issues that arise when doing research on suicide, all four of these monitoring components are relevant.
This discussion of monitoring will refer to three distinct types of oversight: medical monitoring, GCP monitoring, and Data and Safety Monitoring Board (DSMB) oversight. The study design, size (sample size/number of sites), and risks presented by the population and intervention or procedures, will determine the specifics of what should be put in place for a given study.
There has been a NIH-wide shift toward greater clinical research stewardship by the funding institution, as well as more emphasis on study monitoring conducted by the research team itself 23 . The expectation of funder and study team monitoring and oversight is applicable to all clinical research. Nevertheless, research involving suicidal or potentially suicidal participants is reviewed carefully by the NIMH to assess the adequacy of data and safety monitoring, particularly plans to protect at-risk individuals.
Concretely, this means that study teams will want to consider whether to include a medical monitor (sometimes referred to as a safety officer) as part of the study staffing, and whether the medical monitor will be independent. The medical monitor role needs to be delineated in the study protocol (i.e., will this person look only at participant-level safety, or will they also review study data to look for study-wide safety trends), and the timeframes defined for medical monitor review of specific events and/or study-wide trends. Additionally, the study team will need to address who will provide the GCP oversight and quality assurance (QA) for the project. This can involve site study staff (or staff from a Data Coordinating Center [DCC] for multi-site trials) tasked with document and data review, or an independent study monitor, typically a Certified Research Associate (CRA). These staffing decisions about both medical monitoring and data monitoring have budgetary implications that are useful to have in mind during project planning.
In addition to internal data and safety monitoring, clinical trials involving suicide are often expected to have an independent DSMB whose function is to provide independent oversight to assure data integrity and that safety trends are being evaluated. While IRBs review matters of ethics, safety, and protocol integrity from individual research sites, DSMBs tend to focus on data integrity and study-wide safety trends that might only become evident when aggregating and comparing data across study arms or in study subgroups, and in particular across study sites in the case of multi-site studies. Typically, adverse events and serious adverse events are reported to DSMBs, as well as data on study progress and protocol integrity to assure that the study is moving forward as planned and that the data are robust so that research questions can be answered. For assistance determining the appropriate level of monitoring for a specific study and guidance on developing a data and safety monitoring plan for an NIMH-funded study, investigators can refer to the NIMH Clinical Research webpage. The NIMH also has a Clinical Research Toolbox that includes NIH and NIMH policy and guidance documents related to human subject research, as well as templates, sample forms, and other materials to support clinical investigators in the development and conduct of high-quality clinical research studies.
Another aspect of safety monitoring for investigators to consider involves developing appropriate study procedures for assessing and addressing clinical worsening and suicidal crises. Depending on the research design, these issues are more or less likely to be detected, and investigators will want to think about the risk ascertainment opportunities presented in their study design, as well as potential ascertainment biases (e.g., interventions with more patient contact may have more opportunities to detect risk status than control arms). In addition to established in-person or remote assessments of suicide risk, the emergence of real-time data capture and transfer provides researchers with the opportunity to attempt to monitor clinical states in real-time using ecological momentary analysis (EMA) and other forms of active and passive data capture, to examine daily fluctuations in suicidal ideation and/or intent that also incorporate assessment of clinical worsening. Decisions for when and how to address clinical worsening in real-time should be considered during the development of the study protocol and informed consent.
Methodologically, study teams assessing clinical worsening and suicide risk will need to determine the timeframes to consider in these assessments. For example, should the study participants’ clinical state over the course of the study be compared to their previous assessment, or to their baseline status at the outset of the study? Also, is status at a given assessment point going to be based on what they report at a given moment in time (as is captured with EMA), or on what they report has occurred during a specific time period? The answers to these questions will depend on many methodological and study design factors, as well as the purpose of the data being collected (i.e., risk management versus outcome data). Thus, when developing safety monitoring protocols, numerous questions arise for investigators to address:
What events are appropriate to consider as expected and unexpected adverse events and serious adverse events for the study?
Suicide-related clinical research studies are unique in that adverse and serious adverse events involving suicidal ideation and behavior are expected to occur. Moreover, these are often the primary study endpoints. In some cases, the safety monitoring protocols encouraged in studies with suicidal participants make it more likely that these events, and clinical worsening, will be identified and reported to designated funding agencies and regulatory bodies. Reporting of AEs to the various governmental agencies and non-governmental entities that provide regulatory oversight or funding to a study can present challenges to study teams due to the variability in definitions of AEs and reporting expectations utilized by different federal funding agencies, governmental regulatory agencies, and non-governmental regulatory entities 23 . NIH follows OHRP guidance on the definition of AEs and associated reporting expectations. In addition, NIMH has a reportable events policy that delineates timeframes for reporting events to the NIMH. Suicide prevention studies also present a unique circumstance for the reporting of SAEs because suicidal behavior (ideation, attempts, and suicide deaths) are both expected occurrences and primary study outcomes.
As noted above, there is variability in the field and among regulatory bodies on the definition of AEs and SAEs, specifically about whether expectedness and relatedness should bear on whether an event is considered an SAE. Oquendo, et al. (2011) 14 looked at the issue of variability in the definition and reporting of adverse event in suicide prevention trials and recommended that the definition on an AE and SAE stand independent of assessments of relatedness and expectedness. As there continues to be inconsistency in the field, however, it is critical that study protocols provide clear definitions of what will be considered AEs and SAEs and ensure they have procedures in place to collect information regarding these events. In addition, specifying what events (i.e., non-suicidal self-injury, suicidal ideation, suicide attempt, hospitalization, suicide deaths) will be expected AEs and SAEs in the study determines parameters for IRB and DSMB reporting.
In psychiatric medication studies, suicide attempts are typically considered unexpected and immediately reportable SAEs regardless of relatedness, and thus follow immediately reportable guidelines specified by regulatory bodies overseeing the study. For suicide research studies, given that these SAEs are expected, the reporting guidelines may be different for expected and unrelated suicide ideation and attempts, i.e., these events might not be deemed immediately reportable. Delineating in the study protocol what SAEs are immediately reportable based on institutional, state, and federal regulations and determinations of expectedness and relatedness is necessary. The protocol then provides needed guidance to study staff, as well as to funding and regulatory bodies providing study oversight. A final aspect to include in a research protocol is the AE and SAE collection, documentation and reporting procedures, specifying both the staff workflow and timeline for reporting to relevant funding and regulatory bodies.
What unique issues arise in monitoring and reporting in multi-site trials?
Data and safety monitoring and reporting challenges arise for multi-site studies which need to coordinate these efforts across sites and establish effective data quality and management procedures. Reporting expectations for NIH multi-site trials involve providing data to a DSMB, as NIH-funded multi-site clinical trials which involve interventions that present potential risk to participants are required to have an independent DSMB, or in some cases, one constituted by the NIH (NOT-98-084 ). These monitoring and reporting expectations are not unique to research on suicide, but their importance is underscored when the safety data being collected, reviewed, and reported are related to assessments of suicidal ideation and behavior.
A fundamental challenge that multi-site trials must address is establishing and implementing parallel monitoring and reporting processes across sites. Cross-site synchrony can be addressed by having centralized study staff providing site good clinical practice (GCP) monitoring, or medical monitoring. This centralized strategy for medical monitoring mitigates potential discrepancies in approach or implementation across sites for participant continuation determinations, clinical worsening assessments, and adverse and serious adverse event determinations. On the other hand, some studies opt for site-based medical monitors, as research teams want to have a medical monitor available to be part of the clinical assessment process for determinations of clinical worsening and study continuation. Medical monitors in multi-site trials often play dual roles, both making safety determinations for individual patients, and also looking at study-wide safety trends. Thus, if medical monitors are site-based, plans should be made for looking at study wide safety data, both to look for trends, as well as site variability issues. Similarly, for GCP monitoring, some studies opt to have site-based monitors implementing a study-wide monitoring plan, while others have study-wide monitors. Again, studies with site-based GCP monitoring should periodically review study-wide GCP compliance and site variability. Whether in a single-site or multi-site trial, decisions about how to structure the GCP and medical monitoring will be driven by issues of study design and patient population, as well as by available resources. As noted earlier, budgeting for adequate GCP and medical monitoring is appropriate for NIMH-funded studies.
What risk management and data and safety monitoring elements are appropriate to include in non-intervention studies?
Suicide research can include a wide-range of studies attempting to better understand risk and prevention. For example, the fields of psychiatry and neuroscience are engaged in research efforts to identify biomarkers that indicate elevated risk for suicide, as well as the mechanisms of action by which potential treatments have their effect 25 . More broadly, risk prediction studies are utilizing patient data from large government and private health care systems databases 26 , 27 , high volume passive and active real-time data from individuals at elevated risk 28 , as well as social media data 29 . Also, as noted above, more non-suicide focused observational mental health research studies are asking questions about suicide in assessments.
Although many of these studies do not involve intervention research, they need data and safety monitoring plans. The level of monitoring indicated for clinical research studies should be based on the risks presented by the participant population and the study procedures, as discussed in the NIMH Guidance on Risk Based Monitoring. Clinical research studies that include currently or recently suicidal participants will need plans for participant monitoring, risk management, and adverse events documentation and reporting that are in line with the study design. In some cases, an independent safety monitor or a DSMB would be appropriate for a non-intervention study; this would be influenced by the number of participants and sites, the length of study participation, the risks presented by the subject population, and the complexity and risks of the study procedures.
Researchers who include questions about suicide in research studies with populations not identified as being at elevated risk will identify information about suicide ideation and behavior in the study population. These study teams will need procedures for responding to the identified risk and providing clinical follow up as necessary, as well as for adverse event reporting. The Adolescent Brain and Cognitive Development (ABCD) study, a large longitudinal study following 9 and 10 year-olds into early adulthood, includes an assessment asking participating children and their parents about suicide. In Barch et al.’s (2018) discussion of the ABCD study’s mental health assessments, the investigators note that each participating site developed a protocol detailing how the site team would respond if a suicide-related event were disclosed by the children or their parents 30 . The experience of the ABCD study will provide lessons on what support mental health researchers need as preparation to include questions about suicide risk in their assessments. Non-suicide researchers may need guidance on how to provide appropriate follow up, given their study design and staffing, to individuals who endorse some level of risk (e.g., what level of follow up is needed? what staffing and training are required to provide that follow up?). Regulatory bodies, such as IRBs, may also benefit from guidance on the topic of how to manage suicide risk in the context of a study that may neither be focused on suicide, nor include individuals at known risk of suicide (e.g., survey research that is not suicide focused, but opts to include questions relevant to suicide risk). Involvement or consultation of experienced clinician researchers may be helpful to think through issues such as appropriate use of crisis lines, when to facilitate assessments and referrals, and/or when study staff trained to do a suicide risk evaluation are needed 31 .
What study procedures need to be in place to respond to suicidal crises and clinical worsening that occur during the study?
Studies that include suicidal and potentially suicidal participants have developed a range of clinical management options appropriate for their study designs (i.e., in-person and remote risk assessments). This involves the safety monitoring referred to above, and then planning steps to attempt to connect participants with appropriate care. Study protocols should provide a detailed description of this process and its staffing (i.e., the provision of clinical backup for non-clinical staff conducting assessments). Some studies in which the intervention or rater sessions are completed remotely--and often by raters who are not clinicians--have developed clinical management protocols that include warm transfers to crisis lines (i.e., making a telephone connection to a crisis resource while maintaining contact with the participant in order to make a direct transfer). These protocols require establishing arrangements that set up the technical and contractual links necessary to do these transfers, training study staff, setting up the privacy and confidentiality protections that allow such information sharing to take place, and monitoring to ensure that all precautions are implemented as planned throughout the study. When high risk youth participants are involved in a study, there may be an additional step of informing parents and guardians about the crisis and the need for intervention. Whether for adults or youth, these risk management procedures need to be shared with participants as part of the informed consent (and assent) process.
Another aspect of clinical management arises when participant symptoms worsen, and individuals need care outside the study protocol before further participation in the research. Determination of clinical worsening or suicidality can trigger early termination from the study. In such cases, study teams will want to think through how to manage potential conflicts of interest that arise for researchers when an assessment of clinical deterioration could lead to loss of a study participant. Specifically, what internal and independent clinical review and oversight processes need to be in place to assure patient safety in cases when a change in a study participant’s clinical status determines ongoing research participation? Some studies, based on study design and risks presented, will benefit from an independent medical monitor in an oversight role to conduct clinical assessments as part of the evaluation of study continuation, and/or to review the documentation of the assessment and the study continuation decision made by a member of the study team. A DSMB can also play a role in this oversight, to assure that ethical decisions are being made with regard to ongoing study participation. Documentation of the decision-making process regarding risk management and study continuation allows study monitors and DSMBs to assure that study teams are conducting studies safely, ethically, and as per protocol.
What are the issues to anticipate and plan for when participants come to the end of their study participation due to early termination, study completion, or a participant death?
Studies which enroll participants endorsing recent/active suicidal ideation and behavior need plans in place for the end of study participation as it impacts patient safety and welfare, most importantly in cases when individuals are not engaged in ongoing care. Two issues that need attention include end of study procedures: 1) to assess that individuals are safe and have information that will help them stay safe (e.g., crisis line numbers, clinic resources), and relatedly, 2) assuring that individual are not being “abandoned” without care by providing linkages to care when needed. Because many questions remain regarding the longer-term benefit of interventions, investigators may also wish to invite those terminating the initial study early to continue to participate in follow-up assessments, and/or allow the review of their records from health systems or other systems relevant to the study population (i.e., justice, schools) so that outcome data can continue to be collected.
In the case of a patient death, what protocol is in place for contact with the family, and support of research staff?
While suicide deaths in research studies are rare, individuals at risk for suicide are at elevated risk for multiple causes of mortality 32 , 33 . Study participant deaths are a critical topic that study teams need to plan for in their study procedures, and suicide deaths, in particular, may require additional consideration. As noted in the adverse and serious adverse events reporting discussion above, protocols should include language on whether suicide deaths are considered expected or unexpected adverse events, what oversight bodies need to receive serious adverse event reports, and on what timeline. Guidance can include planning who on the study team will speak with family members and, when appropriate, other study participants; how to talk about the suicide with these affected individuals; and what the process will be for providing supportive resources to family members and study team members.
What issues and resources are helpful to keep in mind when bringing suicide-relevant studies to oversight boards for regulatory review?
One frequently cited reason for excluding individuals at elevated risk of suicide in a study, or for not including questions about suicide in a research assessment protocol, is the reluctance of some oversight bodies to approve these studies based on concerns about participant safety and potential for harm 34 . Andriesses, et al. (2019) underscore the utility of anticipating an oversight committees’ potential concerns and engaging in early discussions to address ethical questions. Common issues raised by oversight committees include: the potential iatrogenic effects of participation in research on suicide, whether effective clinical monitoring and safety planning strategies are in place for responding to emergent suicide risk during study participation, and, if relevant, whether evidence-based pharmacological or psychosocial treatments for reducing the risk of suicide attempt are part of the study protocol.
Recent published work examining the potential iatrogenic effects of participating in suicide research supports Gould, et al.’s (2005) 35 findings that suicide screening programs do not lead to increases in suicidal ideation and distress. A 2018 meta-analysis 36 of thirteen published studies evaluating the iatrogenic effects of assessing suicidality did not find evidence of significant iatrogenic effects. A 2016 study 37 looked at the potential effects of exposure to viewing suicide-related stimuli on the Implicit Association Test (IAT). This research did not find a reliable increase in urges to engage in self-injury and desire to die but did find a moderate mood decline among a subset of the study population. Providing this, or similar, research to oversight bodies may support their ability to evaluate a research protocol that includes suicide assessment or exposure to suicide-related stimuli.
Oversight bodies often have concerns about including people experiencing suicidal thoughts and behaviors in studies and allowing participants to remain in protocols if evidence of clinical worsening related to suicide emerges 38 . Ballard, et al. (2017) provide data on the research participation of depressed individuals experiencing active suicidal ideation (SI) during a study. The team reviewed data from 14 clinical trials which took place at the NIMH Intramural Research Program, looking at the prevalence and stability of the ideation, whether individuals continued in trials if active SI was identified, and participation in procedures of those who continued some level of involvement in the trial with active SI. This work provides support for the viability of trial participation of those with active suicidal ideation. As more studies are conducted with adults, adolescents and children at elevated risk (in addition to studies asking questions about suicide to individuals who are not at known risk), there are a larger number and more varied examples of how to effectively monitor and respond to instances of clinically concerning suicidal thoughts, risk behaviors, and reports of a recent attempt not yet discussed with a mental health provider. One example evaluated by Nuij, et al. (2018) is a mobile self-monitoring and safety planning app for depressed individuals at elevated risk for suicide 39 . Referrals, or warm transfers, to crisis lines represent another clinical management strategy discussed in the literature; Busby, et al. (2019) provide an example from an adolescent suicide screen study 40 .
Providing literature on effective suicide prevention interventions can remind oversight bodies that there are evidence-based treatment strategies that have been shown to reduce suicidal thoughts and behaviors 41 , 42 . In addition, it may be useful to publish lessons learned with the field and share discussions on the ethical and safety challenges presented by this coming generation of studies seeking to develop new approaches to understanding and preventing suicide 13 .
As new clinical designs, methodological approaches, and technological advances have been developed and applied to clinical research, additional considerations have emerged for investigators working with participants at heightened risk of suicide.
What are the unique issues presented by pragmatic trials involving individuals at higher risk for suicide?
Despite a growing number of efficacious treatments that reduce suicide risk 43 , 44 , a limited number of these efficacious interventions have been studied in large-scale community or health delivery settings. The launching of pragmatic effectiveness trials to assess how effective specific prevention interventions are in existing health care systems raises new issues. These trials often have: 1) broad inclusion criteria and few exclusion criteria (e.g., any adult with a given diagnosis, or a given assessment score); 2) automated recruitment and/or enrollment; 3) a modified informed consent process and/or informed consent waivers; 4) care delivered as part of, and within the constraints of, existing health care systems; 5) outcome data extracted from medical records or other administrative health care data sources (i.e., insurance claim data); and 6) large numbers of sites and participants.
Suicide-focused pragmatic trials are likely to receive high levels of regulatory oversight, as they are often large multi-site trials, include a population at elevated risk for suicide, and are of significant public health interest given the hope to impact health policy and practice. At the same time, the research model for pragmatic trials, e.g., learning health care systems 45 differs from efficacy clinical trials that regulatory bodies, such as IRBs and DSMBs, are accustomed to seeing 46 . Working out protocols for informed consent (and obtaining adequate waivers to consent when indicate, risk management, and adverse events reporting may take more back-and-forth between researchers and regulatory bodies than a traditional efficacy trial, as the research study aims and designs are significantly different. For example:
Whereas efficacy trials may include enhancements to care -- including the control condition -- pragmatic trials are less likely to do so. For example, there may be constraints to enhancing the risk assessment and clinical management available in that system. Enhancements to the treatment-as-usual study arm can be limited because study participants receive the care provided in that setting. Finally, how are privacy protections provided when large amounts of patient data are being transferred between administrative health system records and study databases?
Pragmatic effectiveness researchers, ethicists, and regulators are thinking through existing approaches to research ethics and safety to figure out how these apply within a learning health care context with regard to design 49 , consent 47 , and data and safety monitoring 15 , 50 . There are ongoing dialogues among the various stakeholders in this arena of research ethics and practice to address the debates presented by this emerging field 46 , 48 , 51 . The NIH Common Fund is supporting the Health Care Systems Research Collaboratory to explore these questions. The Collaboratory website has numerous relevant resources, including a link to a page of regulatory issues, as well as presentations on issues to think about in data and safety monitoring in clinical trials and Good Clinical Practice .
What are the issues presented by research involving individuals at higher risk for suicide which use newer technologies as a central part of study conduct?
Another emerging area in suicide research involves the use of technology in risk assessment, risk prediction, prevention and treatment, and as the vehicle for delivery of the intervention. Mental health research in general is incorporating technological innovations that are increasingly a standard part of health care delivery and research: cloud-based and mobile technologies and devices, social media, use of EHRs, web portals, secure messaging, and the development of big data platforms are all areas where research is rapidly growing.
In the arena of therapeutic devices, research on evolving electro-convulsive therapy (ECT) and transcranial magnetic stimulation (TMS) protocols, as well as more experimental treatments such as magnetic seizure therapy (MST), are assessing for potential impact on suicidal ideation and behavior. These treatment modalities may have a beneficial effect on suicidal ideation in some patients, including providing rapid anti-suicidality action 52 , 53 , 54 , 55 . In contrast to their approach with psychiatric medications discussed above, the FDA has not issued a guidance with regard to assessment of suicidal ideation and behavior when conducting research using ECT, TMS, and other devices used in the delivery of psychiatric treatment 56 .
For suicide research, a key issue to consider is the conduct of risk assessment, prediction, and management when some or all of the intervention and study contact is remote. Examples include: 1) studies attempting to identify suicide risk in individuals’ electronic communications or social media use; 2) cloud-based interventions that participants engage in through web portals, websites, or apps; and 3) symptom monitoring/care management interventions that tracks patient behavior and symptoms via EMA, smart phone sensors, or responses to automated surveys. In some cases, mental health interventions that use technologies that allow for remote contact and assessment may work to keep study participants more reachable and connected to the study team, or in the case of some pragmatic trials, to the study participants’ clinical providers. Alternatively, participants may be in studies conducted remotely without a study team member to contact in case of a psychiatric emergency, or a study design that involves minimal contact between research staff or clinical providers, and study participants. Traditional models of risk management that rely on in-person or telephone assessment may not be applicable for these studies, and alternative procedures need to be developed. Study teams and technology platforms are working on developing automated strategies for responding in cases where suicidal ideation and/or behavior are endorsed, with considerations about what type of response is needed (e.g., alerts, notifications, crisis line information) and to whom the response should be directed (e.g., providers, medical monitors, study participants, bystanders, those involved in treatment as usual and/or those in an active intervention).
As the field of predictive analytics grows, high volume patient data that could be used for stratification of suicide risk are being ascertained from electronic health records (EHR), ecological momentary assessment (EMA), and social media. The field is progressing to the point that research data is gathered in real time, with the possibility of real-time transmission to research teams. As such, the research community is in the process of developing best practices for data and safety monitoring for these studies. For example, studies that collect EMA or social media data on potentially fluctuating suicidal thoughts and behaviors multiple times a day 57 , 58 face the challenge of establishing a protocol for reviewing safety data, and operationalizing what patterns of response constitute elevated risk or clinical worsening that require some level of action from the study. For non-intervention studies aiming to learn about suicide risk by looking at patterns of thoughts, emotions, and behaviors as they shift in real-time, gathering data on participants’ active and passive responses over time with little or no intervention provides a new way to understand participants’ experiences related to intentional self-injury. Working through the primary ethical obligations to participant safety in the context of this research requires conversations with ethicists, IRBs (and other oversight bodies), legal counsel, and individuals with lived experience to determine best practices for consent language, safety monitoring, and clinical safety protocols 59 . When possible, study teams can conduct exit interviews with participants as part of their pilot work to learn how helpful or effective the strategies used were for informing participants about the study’s approach to suicide risk, as well as to monitoring and responding to assessed risk.
Risk prediction presents an opportunity to significantly improve identification of individuals at risk for suicide attempt, particularly at imminent risk (see 2019 NIMH meeting summary). As there are many complex ethical questions that arise in the context of predictive analytics work when applied to suicide prevention, stakeholders are engaged in discussions of ethical challenges 29 , 60 , 61 , 62 , 63 . These include questions related to informed consent and communication with participants, loss of confidentiality in data transfer and analyses, the implications of demographic biases in the data used to develop the algorithms, the potential negative impact on participants who are identified as being at risk via algorithms, and whether appropriate interventions are available for those participants identified as being at risk. Oversight bodies, ethicists, legal experts, individuals with lived experience, and investigators will benefit from initiating discussions early in project development to develop study protocols and procedures that address ethical and participants safety concerns.
Social media suicide prevention research raises ethical questions for investigators regarding not only the appropriate response to and management of the safety of potentially suicidal study participants, but introduces “by-standers”, those who become aware that someone is at-risk of suicide through an on-line setting (e.g., Facebook and Instagram tools). These are questions that are generating significant interest among researchers as technology represents a rapidly developing area impacting broadly on study conduct in areas such as informed consent, monitoring and reporting, risk management, and data quality and security.
Automated risk assessment and management is increasingly used in studies in which suicide-related outcomes are likely, and, in some cases, are the primary study endpoint. Researchers need to anticipate what can go awry when using electronic methods of ascertaining suicide risk levels. This can include hardware and logistical challenges such as poor cellular phone and Wi-Fi connections. With regard to software, many other problems can occur. For example, if study participant risk is identified via a computer program that processes electronic data based on participant responses to remote assessments, an error or “bug” in the program could interfere with planned safety procedures. Programming errors and bugs can impact the scoring of assessments and notification failures. This issue also applies to electronic methods of identifying participants eligible for study enrollment; programming errors or problems with an electronic algorithm can lead to participants who are eligible for the study not being enrolled or ineligible participants being enrolled. These types of potential problems should be anticipated, and stress tests and other types of regular accuracy checks built into procedures can allow study teams to identify problems and rectify them quickly.
Relatedly, study teams should consider what level of data security they need to build in to their study protocols when the study involves potential exposure of personally identifiable information (PII) and other sensitive information (e.g., suicide risk). Studies that include the use of relatively basic technologies (e.g., texting, email), as well as larger and more complex systems (e.g., sensor data, social media, EHRs, and studies with significant data being transferred between independent databases) will need to work with data security experts during the study planning process.
Finally, researchers developing mobile apps designed for mental health applications will want to consult the FDA’s guidance on mobile medical applications (MMAs); MMAs designed to help individuals with diagnosed psychiatric conditions are on the list of those the FDA has discretion on whether or not to regulate. The FDA provides information on what types of mobile apps the FDA regulates, and examples of mobile apps for which the FDA will exercise enforcement discretion .
As suicide prevention research has advanced to address clinical course, risk detection, treatment development, comparative effectiveness, and implementation, the contexts for research have also become more diverse. Researchers 64 and institutions are beginning to share what they have learned regarding safety and ethical procedures. NIMH welcomes this reporting and anticipates further challenges as research progresses. The purpose of this discussion was to provide an overview of issues and approaches for investigators undertaking research with participants at heightened risk of suicide. As is evident, unique ethical and safety considerations arise when doing research with individuals at heightened risk for suicide. Working through these issues when developing a research design and protocol, in consultation with federal policies and guidance, will promote ethically and scientifically robust studies to further the field of suicide research and strengthen suicide prevention efforts.
1 J. Pearson, B. Stanley, C. King and C. Fischer, "Intervention research with persons at high risk for suicidality: safety and ethical considerations," Journal of Clinical Psychiatry, vol. 25, pp. 17-26, 2001.
2 S. Murphy, J. Xu, K. Kochanek and E. Arias, "Mortality in the United States, 2017," Hyattsville, MD, 2018.
3 Substance Abuce and Mental Health Services Administration, "Key substance use and mental health indicators in the United States: Results from the 2017 National Survey on Drug Use and Health," Rockville, MD, 2018.
4 H. Hedegaard, S. Curtin, and M. Warner and H. Hedegaard, "Suicide Mortality in the United States, 1999-2017," Hyattsville, MD, 20186.
5 M. Hogan and J. Grumet, "Suicide Prevention: An Emerging Priority For Health Care," Health Affairs, vol. 35, pp. 1084-1090, 2016.
6 J. Graves, J. Mackelprang, S. Van Natta and C. Holliday, "Suicide Prevention Training: Policies for Health Care Professionals Across the United States as of October 2017," American Journal of Public Health, vol. 108, pp. 760-768, 2018.
7 E. Ballard, M. Pao, L. Horowitz, L. Lee, D. Henderson and D. Rosenstein, "Aftermath of Suicide in the Hospital: Institutional Response," Psychosomatics, vol. 49, no. 6, p. 461–469, 2008.
8 J. Boggs, A. Beck, B. Yarborough, M. Schoenbaum, C. Stewart, J. Richards, G. Simon and B. Ahmedani, "Safety Planning; What does it mean to you?," International Assocaition of Suicide Research, p. Oral Presentation, October 2019.
9 E. Ballard, M. Pao, D. Henderson, L. Lee, J. Bostwick and D. Rosenstein, "Suicide in the Medical Setting," Joint Commission Journal on Quality and Patient Safety / Joint Commission Resources, vol. 34, no. 8, p. 474–481, 2008.
10 Institute for Healthcare Improvement, “Failure Modes and Effects Analysis (FMEA) Tool,” Retrieved from: http://www.ihi.org/resources/Pages/Tools/FailureModesandEffectsAnalysisTool.aspx .
11 R. Clay, "The Cultural Distinctions in Whether, When, and How People Engage in Suicidal Behavior," Monitor on Psychology, pp. 28-35, June 2018.
12 S. MacLean, C. MacKiw and S. Hatcher, "Involving People with Lived Experience in Research on Suicide Prevention," Canadian Medical Association Journal, vol. 190, pp. S13-S14, 2018.
13 R. O'Connor and G. Portzky, "Looking to the Future: A Synthesis of New Developments and Challenges in Suicide Research and Prevention," Frontiers in Psychology, vol. 9, pp. 1-14, 2018.
14 U. Whiteside, J. Richards, D. Huh, R. Hidalgo, R. Nordhauser, A. Wong, X. Zhang, D. Luxton, M. Ellsworth and D. Lezine, "Development and Evaluation of a Web-Based Resource for Suicidal Thoughts: NowMattersNow.org," Journal of Medican Internet Research, vol. 21, no. 5, September 2019.
15 G. Simon, A. Beck, R. Rossom, J. Richards, B. Kirlin, D. King, L. Shulman, E. Ludman, R. Penfold, S. Shortreed and U. Whiteside, "Population-based outreach versus care as usual to prevent suicide attempt: study protocol for a randomized controlled trial," Trials, vol. 17, no. 1, pp. 1-10, 2016.
16 L. Wexler, M. Chandler, J. Gone, Cwik, M, L. Kirmayer, T. LaFromboise, T. Brockie and J. Allen, "Advancing Suicide Prevention Research With Rural American Indian and Alaska Native Populations," American Journal of Public Health, vol. 5, pp. 891-899, 2015.
17 C. Pearson, M. Parker, C. Zhou, C. Donald and C. Fisher, "A culturally tailored research ethics training curriculum for American Indian and Alaska Native communities: a randomized comparison trial," Critical Public Health, vol. 29, pp. 27-39, 2019.
18 C. King and A. Kramer, "Intervention research with youths at elevated risk for suicide: meeting the ethical and regulatory challenges of informed consent and assent," Suicide & Life Threatening Behavior, vol. 5, pp. 486-97, 2008.
19 O. Fawole, S. Dy, R. Wilson, B. Lau, K. Martinez, C. Apostol, V. D, E. Bass and A. R. , "A systematic review of communication quality improvement interventions for patients with advanced and serious illness," Journal of General Internal Medicine, vol. 28, no. 4, pp. 570-577, 2013.
20 R. Kapoor, E. Benedek, R. Bonnie, T. Gandhi, L. Gold, J. Judd and D. Pinals, "Resource Document on Risk-Based Gun Removal Laws," American Psychiatric Association, Washington, DC, 2018.
21 C. Elliott and M. Lamkin, "Restrict the Recruitment of Involuntarily Committed Patients for Psychiatric Research," JAMA Psychiatry, vol. 4, pp. 317-318, 2016.
22 E. Eisenstein, L. Curtis, K. Prather, M. Hogarth, K. Anstrom, D. Gabriel, D. Mark, R. Mentz and S. Greene, "Choosing and Sepcifying Endpoints and Outcomes: Using Death as an Endpoint," [Online]. Available: https://rethinkingclinicaltrials.org/chapters/design/choosing-specifying-end-points-outcomes/using-death-as-an-endpoint/. [Accessed 2019].
23 K. Hudson, M. Lauer and F. Collins, "Toward a New Era of Trust and Transparency in Clinical Trials," Journal of the American Medical Association, vol. 13, pp. 1353-1354, 2016.
24 M. Oquendo, S. Feldman, E. Silverman, D. Currier, G. Brown, D. Chen, P. Chiapella, R. Fischbach, M. Gould, B. Stanley, D. Strauss, J. Zelazny and J. Pearson, "Variability in the definition and reporting of adverse events in suicide prevention trials: an examination of the issues and a proposed solution," Archives of Suicide Research, vol. 1, pp. 29-42, 2011.
25 A. Van Harmelen, L. Schmaal and H. Blumberg, "Journal of Affective Disorders Special Issue on Suicide-Related Research: Hopeful progress but much research urgently needed," Journal of Affective Disorders, vol. 251, pp. 39-41, 2019.
26 R. Kessler, I. Hwang, C. Hoffmire, J. McCarthy, M. Petukkkhova, A. Rosellini, N. Sampson, A. Schneider, P. Bradley, I. Katz, C. Thompson and R. Bossarte, "Developing a practical suicide risk prediction model for targeting high‐risk patients in the Veterans health Administration," International Journal of Methods in Psychiatric Research, vol. 26, pp. 1-7, 2017.
27 G. Simon, E. Johnson, J. Lawrence, R. Rossom, B. Ahmedani, F. Lynch, A. Beck, B. Waitzfelder, R. Ziebell, R. Penfold and S. Shortreed, "Predicting Suicide Attempts and Suicide Deaths Following Outpatient Visits Using Electronic Health Records," American Journal of Psychiatry, vol. 175, pp. 951-960, 2018.
28 N. Allen, B. Nelson, D. Brent and R. Auerbach, "Short-term prediction of suicidal thoughts and behaviors in adolescents: Can recent developments in technology and computational science provide a breakthrough?," Journal of Affective Disorders, vol. 250, pp. 163-169, 2019.
29 G. Coppersmith, R. Leary, P. Crutchley and A. Fine, "Natural Language Processing of Social Media as Screening for Suicide Risk," Biomedical Informatics Insights, vol. 10, pp. 1-11, 2018.
30 D. Barch, M. Albaugh, S. Avenevoli, L. Chang, D. CLark, M. Glantz, J. Hudziak, T. Jernigan, S. Tapert, D. Yurgelun-Todd, N. Alia-Klein, A. Potter, M. Paulus, D. Prouty, R. Zucker and K. Sher, "Demographic, physical and mental health assessments in the adolescentbrain and cognitive development study: Rationale and description," Developmental Cognitive Neuroscience, vol. 32, pp. 55-66, 2018.
31 C. King, "Asking or not asking about suicidal thoughts," American Psychological Association Monitor on Psychology, vol. 7, p. 36, 2016.
32 I. Katz, T. Peltzman, J. Jedele and J. McCarthy, "Critical Periods for Increased Mortality After Discharge From Inpatient Mental Health Units: Opportunities for Prevention," Psychiatric Services, vol. 70, no. 6, pp. 450-456, 2019.
33 P. Batterham, A. Calear, A. Mackinnon and H. Christensen, "The association between suicidal ideation and increased mortality from natural causes," Journal of Affective Disorders, vol. 150, no. 3, pp. 855-60, 2013.
34 K. Andriesses, L. Reifels, K. Krysinka, J. Robinson, G. Dempster and J. Pirkis, "Ethical Concerns in Suicide Research: Results of an International Researcher Survey," Journal of Empirical Research on Human Research Ethics, July 2019.
35 M. Gould, F. Marrocco, M. Keinman, J. Thomas, K. Mostkoff, J. Cote and M. Davies, "Evaluating Iatrogenic Risk of Youth Suicide Screening Programs: A Randomized Controlled Trial," Journal of the American Medical Association, vol. 293, pp. 1635-1643, 2005.
36 C. DeCou and M. Schumann, "On the Iatrogenic Risk of Assessing Suicidality:A Meta-Analysis," Suicide and Life-Threatening Behavior, vol. 48, pp. 531-544, 2018.
37 C. Cha, G. JJ, D. C. E. D’Angelo, J. Hooley, B. Teachman and M. Nock, "Examining Potential Iatrogenic Effects of Viewing Suicide and Self-Injury Stimuli," Psychological Assessment, vol. 28, pp. 1510-1515, 2016.
38 E. Ballard, S. Snider, A. Nugent, D. Luckenbaugh, L. Park and C. Zarate, "Active suicidal ideation during clinical antidepressant trials," Psychiatry Research, vol. 257, pp. 303-308, 2017.
39 C. Nuij, W. van Ballegooijen, J. Ruwaard, D. de Beurs, J. Mokkenstorm, E. van Duijn, R. de Winter, R. O'Connor, J. Smit, H. Riper and A. Kerkof, "Smartphone-based safety planning and self-monitoring for suicidal patients: rationale and study protocol of the CASPAR (Continuous Assessment for Suicide Prevention And Research) study," Internet Interventions, vol. 13, pp. 16-23, 2018.
40 D. Busby, C. King, D. Brent, J. Grupp-Phelan, M. Gould, K. Page and T. Casper, "Adolescents’ Engagement with Crisis HotlineRisk-management Services: A Report from theEmergency Department Screen for TeenSuicide Risk (ED-STARS) Study," Suicide and Life-Threatening Behavior, May 2019.
41 G. Brown and S. Jager-Hyman, "Evidence-Based Psychotherapies for Suicide Prevention: Future Directions," American Journal of Preventive Medicine, vol. 4, p. S186–S194, 2014.
42 E. Hofstra, C. van Nieuwenhuizen, M. Bakker, D. Ozgul, I. Elfeddali, S. de JOng and C. van der Feltz-Cornelis, "Effectiveness of suicide prevention interventions: A systematic review and meta-analysis," General Hospital Psychiatry, May 2019.
43 E. O’Connor, B. Gaynes, B. Burda, C. Soh and E. Whitlock, "Screening for and treatment of suicide risk relevant to primary care: a systematic review for the U.S. Preventive Services Task Force," Annals of Internal Medicine, vol. 158, p. 741–54, 2013.
44 G. Zalsman, K. Hawton, D. Wasserman, K. Heeringen, E. Arensman, M. Sarchiapone and V. J. Carli, "Suicide prevention strategies revisited: 10-year systematic review," The Lancet Psychiatry, vol. 7, p. 646 – 659, 2016.
45 Institute of Medicine, "Best care at lower cost: The path to continuously learning health care in America.," The National Academies Press, Washington, DC, 2013.
46 R. Faden, N. Kass, S. Goodman, P. Pronovost, S. Tunis and T. Beauchamp, "An ethics framework for a learning health care system: a departure from traditional research ethics and clinical ethics," 2013.
47 S. Kim and F. Miller, "Informed Consent for Pragmatic Trials — The Integrated Consent Model," New England Journal of Medicine, vol. 8, pp. 769-772, 2014.
48 S. Chen and S. Kim, "A framework for analysis of research risks and benefits to participants in standard of care pragmatic clinical trials," Clinical Trials, vol. 6, pp. 605-611, 2016.
49 J. Flory, A. Mushlin and Z. Goodman, "Proposals to Conduct Randomized Controlled Trials Without Informed Consent: a Narrative Review," Journal of General Internal Medicine, pp. 1-8, 2016.
50 S. Ellenberg, R. Culbertson, D. Gillen, G. S. S. Schrandt and M. Zirkle, "Data monitoring committees for pragmatic clinical trials," Clinical Trials, vol. 5, pp. 530-6, 2015.
51 S. Kim and F. Miller, "Waivers and Alterations to Consent in Pragmatic Clinical Trials: Respecting the Principle of Respect for Persons," IRB: Ethics & Human Research, vol. 1, pp. 1-5, 2016.
52 M. Fink, C. Kellner and W. McCall, "Commentary: The role of ECT in suicide prevention," Journal of ECT, vol. 30, pp. 5-9, 2014.
53 M. George, R. Raman, D. Benedek, C. Pelic, G. Grammer, K. Stokes, M. Schmidt, C. Spiegel, N. Dealmeida, K. Beaver, J. Borckardt, X. Sun, S. Jain and S. M. , "A two-site pilot randomized 3 day trial of high dose left prefrontal repetitive transcranial magnetic stimulation (rTMS) for suicidal inpatients," Brain Stimulation, vol. 7, pp. 421-431, 2014.
54 M. Pinna, M. Manchia, R. Oppo, F. Scano, G. Pillai, A. Loche, P. Salis and G. Minnai, "Clinical and biological predictors of response to electroconvulsive therapy (ECT): A review," Neuroscience Letters, vol. 16, pp. 30801-1, 2016.
55 Y. Sun, F. Farzan, B. Mulsant, T. Rajji, P. Fitzgerald, M. Barr, J. Downar, W. Wong, D. Blumberger and Z. Daskalakis, "Indicators for remission of suicidal ideation following magnetic seizure therapy in patients with treatment-resistant depression," JAMA Psychiatry, vol. 73, pp. 337-345, 2016.
56 M. Rudorfer, Personal communication, 2016.
57 E. Kleiman, B. Turner, S. Fedor, E. Beale, J. Huffman and M. Nock, "Examination of Real-Time Fluctuations in Suicidal Ideation and Its Risk Factors: Results From Two Ecological Momentary Assessment Studies," Journal of Abnormal Psychology, vol. 126, p. 726–738, 2017.
58 E. Kleiman, M. AJ, V. Joyce, C. Nash, R. Buonopane and M. Nock, "Feasibility and acceptability of using wearable physiological monitors with suicidal adolescent inpatients," JMIR mHealth and uHealth, forthcoming/in press.
59 M. Nock, E. Kleiman, M. Abraham, K. Bentley, D. Brent, R. Buonopane, F. Castro‐Ramirez, C. Cha, W. Dempsey, J. Draper, C. Glenn, J. Harkavy‐Friedman, M. Hollander, J. Huffman, H. Lee, A. Millner, D. Mou, J. Onnela, R. Picard, H. Quay, O. Rankin, S. Sewards, J. Torous, J. Wheelis, U. Whiteside, G. Siegel, A. Ordóñez, J. Pearson, “Consensus Statement on Ethical & Safety Practices for Conducting Digital Monitoring Studies with People at Risk of Suicide and Related Behaviors,” Psych Res Clin Pract. xx:0, 2020.
60 N. Gomes de Andrade, D. Pawson, D. Miruello, L. Donahue and J. Guadagno, "Ethics and Artificial Intelligence: Suicide Prevention on Facebook," Philosophy & Technology, vol. 31, p. 669–684, 2018.
61 K. Linthicum, K. Schafer and J. Ribeiro, "Machine learning in suicide science: Applications and ethics," Behavioral Science and the Law, vol. 37, p. 214–222, 2019.
62 L. McKernan, E. Clayton and C. Walsh, "Protecting Life While Preserving Liberty: Ethical Recommendations for Suicide Prevention With Artificial Intelligence," Frontiers in Psychiatry, vol. 9, p. 650, 2018.
63 R. Tucker, M. Tackett, D. Glickman and M. Reger, "Ethical and Practical Considerations in the Use of a Predictive Model to Trigger Suicide Prevention Interventions in Healthcare Settings," Suicide and Life Threatening Behavior, vol. 49, pp. 382-392, 2019.
64 W. McCall, R. Benca, P. Rosenquist, M. Riley, C. Hodges, B. Gubosh, L. McCloud, J. Newman, D. Case, M. Rumble, M. Mayo, K. White, M. Phillips and A. Krystal, "A Multi-Site Randomized Clinical Trial to Reduce Suicidal Ideation in Suicidal Adult Outpatients with Major Depressive Disorder: Development of a Methodology to Enhance Safety," Clinical Trials, vol. 3, pp. 189-198, 2015.