Tag: Analytic methods

Evaluating Zero Suicide Care Improvement Programs in MHRN Health Systems

Grant Details

Title: An Evaluation of the National Zero Suicide Model Across Learning Healthcare Systems

Funder: NIMH

Grant Number: 1U01MH114087

Grant Period: 08/03/2017 – 05/31/2022

Narrative: Health systems at six participating sites have all committed to developing and implementing various components of a National Zero Suicide Model (NZSM), originally developed at the lead site for this study, Henry Ford Health System (HFHS).  Each health system will decide which components to implement at their respective site.  This study will develop metrics to measure fidelity and outcomes for the NZSM components implemented in each system using EHR and insurance claims data.  The project will then use these metrics to conduct fidelity and outcome evaluation of the various NZSM approaches in each system using an Interrupted Time Series Design.

Short-term project objectives:

We seek to accomplish three specific aims:

  1. Collaborate with health system leaders across sites to develop EHR metrics to measure specific quality improvement targets and care processes tailored to local NZSM implementation.
  2. Examine the fidelity of the specific NZSM care processes implemented in each system.
  3. Investigate suicide attempt and mortality outcomes within and across NZSM system models.

Long-term project objectives:

Learnings from this study will be immediately available on the Zero Suicide and MHRN websites, shared directly with SAMHSA and NIMH (thru the MHRN), and disseminated broadly to health systems via Zero Suicide Training Academies well before published data are available. As such, our goal is rapid dissemination and translation to practice, as opposed to the standard research-to-practice model – which the NIH and others estimate can take 17 years.

  • Lead Site:
    • Overall PI: HFHS Brian Ahmedani
  • Participating Sites/Subcontractors:
    • KPWA (site PI Greg Simon)
    • KPCO (site PI Jennifer Boggs)
    • KPNW (site PI Greg Clarke)
  • Funder Contacts
    • Science Officer: Susan Azrin
    • Program Official: Michael Freed
    • Grants Management Official: Julie Bergerud

Documents & Reports

Funding Announcement

Personnel Contact List

Publications

Manuscripts in process

ZS manuscript tracker: https://airtable.com/shr7wfbafq5c1rwTY

MHRN manuscript proposal form: https://airtable.com/shrD81CbLqaRrF8ga

MRHN III Supplemental Project: Effect of Initiating Buprenorphine on Suicidal Behavior

Grant Details

Title: Buprenorphine Effect on Suicidal Behavior

Funder: NIMH

Grant Number: U19MH121738-02S2  (supplement to main MHRN cooperative agreement)

Grant Period: 9/17/2020 – 8/31/2022

Narrative: This large observational study will evaluate the effects of initiating buprenorphine treatment on subsequent suicidal behavior among people with opioid use disorder, including those with and without co-occurring mental health conditions or other known risk factors for suicidal behavior. We will use comprehensive health records data from four large health systems serving a combined member/patient population of approximately 11 million. Analyses will examine the overall effect of buprenorphine treatment on subsequent suicide attempts or death, heterogeneity of effects in patient subgroups, and specificity of effects to buprenorphine vs other medications.

We will be using previously developed suicide risk prediction tools to compare the outcomes of individuals who do and do not use buprenorphine with similar baseline suicide risk.

  • Lead Site: KPWA
    • Overall PI: Greg Simon
  • Participating Sites/Subcontractors:
    • KPNC – Site PI: Cynthia Campbell
    • KPSC – Site PI: Rulin Hechter
    • Henry Ford – Site PI: Brian Ahmedani
  • Funder Contacts:
    • Science Officer: Susan Azrin
    • Program Official: Michael Freed
    • Grants Management Official: Julie Bergerud
  • Awarded Budget (Total Cost):
    • Year 1: $514,616
    • Year 2: $274,321

Documents

Funding Announcement: PA-18-591

Notice of Award

Personnel Contact List

Human Subjects: YES

  • IRB Review:
    • KPWA is single IRB reviewing for KPWA, KPNC, and KPSC – Approved waiver of consent for use of records data
    • Henry Ford determined to be exempt
    • KPWA IRBnet File: [1649129]

Clinical Trial: NO

Current status

Exploratory analyses (in preparation for extraction of data at each site) have examined availability and quality of data regarding opioid medication use, availability and quality of data regarding injury and poisoning events, and types of visits occurring prior to initial buprenorphine prescriptions.  These analyses are informing refinements to research design and data specifications.  Final data extraction will occur during in October 2022 with analyses complete in early 2023.

Summary of findings

Not yet available

Publications

None

Research Design and Analytic Methods Interest Group

Goals/Mission

The Research Design and Analytic Methods Interest Group provides a forum for investigators and analysts at MHRN research centers as well as external collaborators to share expertise, receive consultation, and identify questions for future methods development or methodologic research.

Research Areas

  • Design and analysis in pragmatic clinical trials
  • Statistical learning and prediction using health records data
  • Causal inference in observational studies using health records data
  • Privacy protection and re-identification risk

Contact

Susan M Shortreed, PhD

Investigator

Kaiser Permanente Washington Health Research Institute

1730 Minor Ave.  #1600

Seattle, WA  98101

Phone: 206-287-2900

Email: Susan.M.Shortreed@kp.org

Computational Modeling of Suicide Risk

Project Name:
Computational Modeling of Suicide Risk
Principal Investigator:
Gregory Simon MD MPH
Principal Investigator Contact Information: 
gregory.e.simon@kp.org
Principal Investigator Institution:
KP Washington Health Research Institute
Funder:
National Institute of Mental Health (NIMH)
Funding Period:
7/1/2017 to 6/30/2019
Abstract:
A previous supplement to the MHRN cooperative agreement supported development of a population-based suicide risk calculator, predicting risk of suicide attempt or suicide death following an outpatient visit using both responses to PHQ9 item 9 and discrete data extracted from health system electronic health records.  Using a database of approximately 20 million visits by 3 million patients aged 13 and over, we developed and validated machine learning logistic regression models predicting risk of suicide attempt and suicide death within 90 days of an outpatient visit – either a visit to a specialty mental provider or a primary care visit in which a mental health diagnosis was recorded.  Potential predictors included demographic and clinical data extracted from health system records for the 5 years prior to each visit: prior suicidal behavior, mental health and substance use diagnoses, general medical diagnoses, prescriptions for psychiatric medications, inpatient or emergency department mental health care, and responses to routinely administered PHQ9 depression questionnaires.  Models were developed in a 65% random sample of visits and validated in the remaining 35%.  Variable selection models considered 150 discrete predictors and 164 potential interactions.  In the validation sample, the 5% of mental health specialty visits with highest risk scores accounted for 43% of subsequent suicide attempts and 47% of suicide deaths.  Areas under the receiver operating characteristic curves (AUCs) for prediction of suicide attempt and suicide death were 0.85 and 0.86.  In the validation sample, the 5% of primary care visits with highest risk scores accounted for 48% of subsequent suicide attempts and 43% of suicide deaths.  AUCs for prediction of suicide attempt and suicide death were 0.85 and 0.83.

While these models represent a substantial advance over existing risk prediction or risk stratification tools, we identify several significant limitations.  Fixed limits of our computational methods (penalized LASSO logistic regression in the R computing environment) forced us to limit both our sample size and the number of potential predictors and interaction terms.  Those methods also limit ability to appropriate account for clustering of observations within patients and account for the sparse and skewed distributions of predictor data.  Finally, we now recognize the need to extend these methods to predict risk following acute-care (inpatient and emergency department) encounters. We now propose a next stage of work to address these limitations.  Specific aims of this next stage include: Expand and enhance the risk prediction dataset to: include larger numbers of observations with data regarding self-reported suicidal ideation (PHQ9 Item 9), include additional encounters and events following the transition from ICD9 to ICD10 diagnoses, and allow more detailed consideration of the timing of predictor events (diagnoses, encounters, prescription fills)Expand sampling to include emergency department and inpatient encounters. Evaluate alternative modeling approaches, including classification- or tree-based approaches such as Classification and Regression Trees (CART), Mixed Effects Regression Trees (MERT), and Random Forest. Rapidly disseminate all methods, tools and results to a wide range of stakeholders including health systems, researchers, and EHR vendors.
Grant Number:
MH 092201 (supplement)
Participating Sites:
Kaiser Permanente Washington
Kaiser Permanente Northwest
Kaiser Permanente Southern California
Kaiser Permanente Hawaii
HealthPartners
Henry Ford Health System:
Kaiser Permanente Colorado
Investigators:
Gregory Simon MD MPH
Susan Shortreed PhD
Yates Coley PhD
Frances Lynch PhD
Jean Lawrence ScD
Beth Waitzfelder PhD
Rebecca Rossom MD MS
Brian Ahmedani PhD
Arne Beck PhD
Major GoalsExpand and enhance the risk prediction dataset to: include larger numbers of observations with data regarding self-reported suicidal ideation (PHQ9 Item 9), include additional encounters and events following the transition from ICD9 to ICD10 diagnoses, and allow more detailed consideration of the timing of predictor events (diagnoses, encounters, prescription fills)Expand sampling to include emergency department and inpatient encounters. Evaluate alternative modeling approaches, including classification- or tree-based approaches such as Classification and Regression Trees (CART), Mixed Effects Regression Trees (MERT), and Random Forest. Rapidly disseminate all methods, tools and results to a wide range of stakeholders including health systems, researchers, and EHR vendors.
Description of study sample:
Expected to include approximately 30 million encounters by approximately 4 million members in seven health systems.
Current Status:
As of 7/1/2019:  Data harvest, data quality control, and preliminary analyses are complete for the mental health and primary care visit cohorts. Data harvest and data quality control are underway for inpatient and emergency department cohorts.
Study Registration:
N/A
Publications:
N/A
Resources:
N/A
Lessons Learned:
Preliminary analyses indicate that: Models developed prior to October 2015 to predict ICD-9 self-harm diagnoses appear to perform as well when used after October 2015 to predict ICD-10 self-harm diagnoses. More complex ensemble-based model development methods (such as Random Forests) do not appear superior to parametric (such as penalized logistic) methods when predictors are primarily dichotomous. Inclusion of multiple visits per patient does not appear to contribute to over-fitting with parametric model development methods.
What’s next?
N/A

Diversity Supplement – Understanding factors that lead to disparities in depression treatment

Project Name:
Diversity Supplement – Understanding factors that lead to disparities in depression treatment
Principal Investigator:
Karen J Coleman, PhD
Principal Investigator Contact Information:
Karen.J.Coleman@kp.org
Principal Investigator institution:
Kaiser Permanente Southern California
Funder
NIMH
Funding Period:  
09/2014 – 06/2016 (no-cost extension through 06/2017)
Abstract:
Depression and other mental illnesses lead to more disability than the most prevalent physical chronic illnesses such as heart disease, diabetes, and cancer, and may cost the U.S. healthcare system as much as 300 billion dollars annually. There are clear racial and ethnic differences in depression treatment, however, it is unknown if these are patient, provider, or healthcare system driven. The diversity supplement was designed to build on previous work funded within the Mental Health Research Network (MHRN) on practice variation in the treatment of depression. The original aims of the diversity supplement were as follows: Aim 1: To understand the healthcare system-, provider-, and patient-level factors that predict taking the initial antidepressant medication prescribed and/or attendance at the initial psychotherapy visit (primary adherence) within 30 days of an initial depression diagnosis.
Aim 2: To identify the healthcare system-, provider-, and patient-level factors that predict continuation of depression-related treatment once started (secondary adherence).
AIM 3: To characterize racial/ethnic disparities in the achievement of depression improvement or remission with treatment as assessed with the patient health questionnaire (PHQ9), and to understand the role of adherence in this response to treatment.
Grant Number:  
U19MH092201 (Supplement under MHRN II)
Participating Sites Contributing Data:
Kaiser Permanente Southern California, Pasadena, CA
Group Health Cooperative, Seattle, Washington
HealthPartners Institute, Minneapolis, Minnesota
Kaiser Permanente Colorado, Denver, Colorado
Kaiser Permanente Hawaii, Honolulu, Hawaii
Henry Ford Healthcare Systems, Detroit, Michigan
Additional Sites Participating in the Study:
Baylor Scott & White, Temple, Texas
University of Utah, Salt Lake City, Utah
Investigators:
Karen J. Coleman, PhD
Gregory Simon, MD MPH
Rebecca Rossom, MD
Arne Beck, PhD
Beth Waitzfelder, PhD
John Zieber, PhD
Brian Ahmedani, PhD
Zach Imel, PhD
Major Goals: To provide a high-level understanding of how race/ethnicity contributes independently to the variation for initiation and continuation of depression treatment. To provide a dataset and documentation associated with this dataset and its analyses that can be used by other researchers interested in the treatment of depression in large healthcare systems. To provide a basis for testing culturally tailored or appropriate interventions that improve the adherence to depression treatment in a variety of patient populations.
Major Limitations: Questions about depression treatment outcomes cannot be addressed with this dataset because PHQ9 data collection in the five healthcare systems during the study period was not widespread. Questions about healthcare system variation in policies and guidelines for depression treatment cannot be addressed with this dataset as these variables were not available for study. Questions about provider-level variation in the treatment of depression can only be addressed for two sites in the study due to the lack of data collected for providers in the other sites. Thus, conclusions about provider-level variation and its contribution to depression treatment modalities and adherence cannot generalize to other healthcare settings.
Description of study sample:
There are two study samples included in this study. One is for initiation of treatment for patients newly diagnosed with depression and one is for adherence to a new episode of antidepressant medication and/or formal psychotherapy treatment in patients diagnosed with depression. Treatment in the Newly Diagnosed Patients 18 and older who had a new depression diagnosis in primary care clinics between 1/1/2009 and 12/31/2013 were included. Patients were excluded if they had a diagnosis of bipolar disorder, schizophrenia spectrum disorder, or other psychosis in the prior two years to the diagnosis date. To ensure the availability of data needed to create the patient sample for all analyses, the sample was limited to those who were continuously enrolled in the healthcare systems for at least 360 days prior to the diagnosis date, allowing a 60-day gap. New episodes of depression were defined as an ICD-9 code for depression made in a primary care setting, with no diagnosis or treatment for depression (either psychotherapy or antidepressant medication) during the 360 days prior to the diagnosis. These patients were followed for 90 days after the diagnosis date to look for the initiation of treatment (see definitions below for treatment). Patients who disenrolled from the healthcare systems in less than 90 days after diagnosis were excluded. Adherence in the Newly Treated
Patients 18 and older who had a new episode of formal psychotherapy treatment (PT) between 1/1/2010 and 12/31/2013 or a new antidepressant treatment (AD) between 1/1/2010 and 12/31/2013 were included. Patients were excluded if they had a diagnosis of bipolar disorder, schizophrenia spectrum disorder, or other psychosis in the prior two years to index date. The sample was also limited to those who were continuously enrolled in the healthcare systems for at least 270 days prior to the index AD/PT episode, allowing a 60-day gap. A new episode of AD/PT treatment was defined as not having any evidence of the same type of treatment (AD or PT) during the previous 270 days before the date of the new episode.  AD episodes with a prescription for trazodone were excluded because this drug is primarily prescribed for sleep disturbance and not depression. We did not consider appointments that were less than 30 minutes and/or clearly designated as only medication management to be formal psychotherapy.
Current Status:
The analytic dataset and its documentation have been compiled.  Further analyses funded by the project are limited to the following manuscripts which are currently in process: The Mental Health Provider as a Source of Racial and Ethnic Disparities in Adherence to Antidepressant Medication and Psychotherapy (Imel et al.)
Study Registration:
N/A
Publications: Coleman KJ, Stewart C, Waitzfelder BE, Zeber JE, Morales LS, Ahmed AT, Ahmedani BK, Beck A, Copeland LA, Cummings JR, Hunkeler EM, Lindberg NM, Lynch F, Lu CY, Owen-Smith AA, Trinacty CM, Whitebird RR, Simon GE. Racial-Ethnic Differences in Psychiatric Diagnoses and Treatment Across 11 Health Care Systems in the Mental Health Research Network. Psychiatr Serv. 2016 Jul 1;67(7):749-57. doi: 10.1176/appi.ps.201500217. Epub 2016 Apr 15.Rossom RC, Shortreed S, Coleman KJ, Beck A, Waitzfelder BE, Stewart C, Ahmedani BK, Zeber JE, Simon GE. Antidepressant adherence across diverse populations and healthcare settings. Depress Anxiety. 2016 Aug;33(8):765-74. doi: 10.1002/da.22532. Epub 2016 Jun 20.Simon GE, Coleman KJ, Waitzfelder BE, Beck A, Rossom RC, Stewart C, Penfold RB. Adjusting Antidepressant Quality Measures for Race and Ethnicity. JAMA Psychiatry. 2015 Oct;72(10):1055-6. doi: 10.1001/jamapsychiatry.2015.1437.Simon GE, Rossom RC, Beck A, Waitzfelder BE, Coleman KJ, Stewart C, Operskalski B, Penfold RB, Shortreed SM. Antidepressants are not overprescribed for mild depression. J Clin Psychiatry. 2015 Dec;76(12):1627-32. doi: 10.4088/JCP.14m09162.Zeber JE, Coleman KJ, Fischer H, Yoon TK, Ahmedani BK, Beck A, Hubley S, Imel ZE, Rossom RC, Shortreed SM, Stewart C, Waitzfelder BE, Simon GE. The impact of race and ethnicity on rates of return to psychotherapy for depression. Depress Anxiety. 2017 Dec;34(12):1157-1163. doi: 10.1002/da.22696. Epub 2017 Nov 2. PubMed PMID: 29095538; PubMed Central PMCID: PMC5718939.Waitzfelder B, Stewart C, Coleman KJ, Rossom R, Ahmedani BK, Beck A, Zeber JE, Daida YG, Trinacty C, Hubley S, Simon GE. Treatment Initiation for New Episodes of Depression in Primary Care Settings. J Gen Intern Med. 2018 Aug;33(8):1283-1291. doi: 10.1007/s11606-017-4297-2. Epub 2018 Feb 8. PubMed PMID: 29423624.
Resources:
A data dictionary and descriptive tables for the data file associated with this project will be available soon. Some research questions cannot be addressed by this dataset and require an initial review and possible discussion to make this determination. For immediate questions, contact Greg Simon at simon.g@ghc.org.
Lessons Learned:
For all systems contributing data to this project, electronic medical records, insurance claims, and other data systems were organized in a Virtual Data Warehouse (VDW) to facilitate population-based research. The VDW is a collection of common data definitions and formats to ensure equivalent de-identified data for analysis. Because the VDW relies on data availability from a diverse set of healthcare settings in the Health Care Systems Research Network customizing data abstraction such as healthcare system policy variables or provider-level descriptive information is difficult and in some cases impossible. This needs to be considered when studies are proposed that examine the interplay of healthcare system-, provider-, and patient-level factors in mental health-related treatment choices and outcomes.
What’s next?
Possible harvest of new PHQ9 data as implementation of screening and treatment follow-up have increased exponentially since 2013. Pursue an R01 to characterize heterogeneity of achievement of depression improvement or remission and incorporate more healthcare sites (only 6 of 13 MHRN sites were included) and use additional provider variation analytic methods. Other possible grant ideas that have been discussed: Culturally-tailored intervention to assist with the decisions around depression treatment (shared decision-making and motivational interviewing models)

Effects of Medical Products on Suicidal Ideation and Behavior

Project Name:
Effects of Medical Products on Suicidal Ideation and Behavior
Principal Investigator:
Gregory Simon, MD, MPH
Principal Investigator Contact Information: 
gregory.e.simon@kp.org
Principal Investigator Institution:
KP Washington Health Research Institute
Funder:
Food and Drug Administration (FDA)
Funding Period:
9/30/2018 to 9/30/2021
Abstract:
We propose a comprehensive program of infrastructure development and methods development to support future generation of real-world evidence addressing these critical gaps.  The project team will include health systems and embedded research organizations with deep expertise in stakeholder engagement, medical informatics, data science, clinical epidemiology, biostatistics, pragmatic clinical trial methods, implementation science, and innovations in care delivery. Specific Tasks include: Augment the existing FDA Sentinel Initiative data infrastructure to support study of severe mental illness, suicidal ideation, and suicidal behavior. Evaluate and improve generalizability of models predicting suicidal behavior for use in future observational research and pragmatic trials. This program will be embedded in 4 integrated health systems serving a combined population of approximately 10 million members.  This work will be conducted in collaboration with health system and patient/family stakeholders, to assure that methods and evidence developed will actually address real-world questions. This infrastructure and methods development will enable a robust program of research regarding the effects of medical products on suicidal ideation and behavior, including: Scalable and re-usable methods to assess suicidal ideation and behavior as an adverse effect of existing products. Scalable and re-usable methods to assess therapeutic effects of existing products for reducing suicidal ideation and behaviorScalable and re-usable methods to rapidly evaluate possible therapeutic and adverse effects of new medical products on suicidal ideation and behavior. Large pragmatic trials to evaluate therapeutic effects of promising new product(s) on suicidal behavior
Grant Number:
N/A
Participating Sites:
Kaiser Permanente Washington
Harvard Pilgrim Healthcare
Kaiser Permanente Northern California
Kaiser Permanente Southern California
Henry Ford Health System                
Investigators:
Gregory Simon MD MPH
Susan Shortreed PhD
Yates Coley PhD
Richard Platt MD MS
Jeffrey Brown PhD
Darren Toh ScD
Jessica Young PhD
Stacy Sterling PhD
Karen Coleman PhD
Jean Lawrence ScD
Brian Ahmedani PhD
Major Goals Augment the existing FDA Sentinel Initiative data infrastructure to support study of severe mental illness, suicidal ideation, and suicidal behavior. Evaluate and improve generalizability of models predicting suicidal behavior for use in future observational research and pragmatic trials.
Description of study sample:
Various analyses are using data regarding approximately 4.5 million members of participating health systems.
Current Status:
Completed data infrastructure work includes:
– A toolkit to assess re-identification risk when sharing data derived from healthcare records:
– More timely updating of mortality data in health system research data warehouses.
– Regular reporting of availability and quality of patient-reported outcome data in health system research data warehouses.
Analyses are complete regarding:
– Value of more detailed data representation and more complex modeling methods for prediction of suicidal behavior.
– Accuracy of ICD-10-CM encounter diagnoses for identifying self-harm events.
– Value of data typically only available from electronic health records for prediction of suicidal behavior.
Study Registration:
N/A
Publications:
Simon GE, Shortreed SM, Boggs JM, Clarke GN, Rossom RC, Richards JE, Beck A, Ahmedani BK, Coleman KJ, Bhakta B, Stewart CC, Sterling S, Schoenbaum M, Coley RY, Stone M, Mosholder AD, Yaseen ZS. Accuracy of ICD-10-CM encounter diagnoses from health records for identifying self-harm events. J Am Med Inform Assoc. 2022 Aug 26:ocac144. doi: 10.1093/jamia/ocac144.
Resources:
N/A
Lessons Learned:
For prediction of suicidal behavior following outpatient mental health visits, more detailed temporal representation and more complex model development methods (random forest or neural networks vs. penalized logistic regression) do not meaningfully improve prediction accuracy.
When using prediction models to account for confounding by indication in observational studies of medication effects on suicidal behavior, random forest models may be slightly – but not meaningfully – superior to penalized logistic regression.
When using health records data to predict suicidal behavior, additional data available only from electronic health records (race, ethnicity, patient-reported outcome results) do not significantly improve prediction over data typically available from insurance claims.
What’s next?
Additional analyses will examine:
– Similarities and differences in prediction of opioid vs. other overdoses
– Similarities and differences in prediction of self-harm vs. accidental overdoses
– Changes in accuracy of suicide risk prediction models with health system implementation of Zero Suicide care improvement programs.

FDA Black Box Warning and Suicide

Project Name:
Longitudinal Analysis of SSRI Warnings and Suicide in Youth
Principal Investigator
Stephen Soumerai, ScD
Principal Investigator Contact Information:          
ssoumerai@hms.harvard.edu
Principal Investigator institution:
Harvard Pilgrim Health Care
Funder
NIMH
Funding Period:
09/10 – 07/2013
Abstract:
Approximately 14-25% of youth experience major depression before adulthood; about 9% of adolescents attempt suicide and 2.9% make a suicide attempt requiring medical attention. Treatment with antidepressant medications has been shown to improve mood and decrease suicidal ideation. However, there has been concern that antidepressants paradoxically increase the risk of suicidal behaviors following initiation of SSRI treatment. The FDA issued several public health advisories and a boxed warning since October of 2003 and, beginning in 2005, all SSRI labeling has required a “black box” warning (BBW) regarding the increased risk of suicidality in children and adolescents taking antidepressants. However, conflicting evidence concerning the true effects of SSRIs on the risk of suicidal behaviors in youth has generated much controversy. Studies following the BBW reported decreased rates of pharmacologic treatment for depression. Another study reported an 18% increase in completed suicides among youth in 2004 and 2005.

This research will contribute to research regarding unintended consequences of regulatory actions. The secondary aim is to assess the utility of sequential analysis for prospectively assessing signals of health policy impacts using the antidepressant warnings as a policy example.
Grant Number:
U19MH092201
Participating Sites:
Harvard Pilgrim Health Care Institute (Lead Site)
Harvard Medical School
Northeastern University
Baylor Scott & White Health jointly with Central Texas Veterans Health Care System
Kaiser Permanente Washington
HealthPartners Institute
Henry Ford Health System
Kaiser Permanente Colorado
Georgia State University
Kaiser Permanente Hawaii
Kaiser Permanente
Kaiser Permanente Northwest
Kaiser Permanente Southern California
University of Tennessee Health Science Center
Harvard Medical School
Brigham and Women’s Hospital
Investigators:
Stephen B. Soumerai, ScD
Christine Y. Lu, PhD
Sengee Toh, ScD
Jessica L. Sturtevant, ScM
Jeanne M. Madden, PhD
Laurel Anne Copeland, PhD
Gregory Simon, MD, MPH
Rebecca Rossom, MD, MS
Brian K. Ahmedani, PhD
Gregory Clarke, PhD
Marsha A. Raebel, PharmD
Ashli Owen-Smith, PhD
Beth Waitzfelder, PhD
Yihe Daida, PhD
Robert Davis, MD, MPH
Stacy Sterling (Enid M. Hunkeler retired), MA, FAHA
Frances Lynch, PhD
Karen J. Coleman, PhD
Robert Penfold
Martin Kulldorff, PhD
Major Goals: Examine the combined effects of FDA warnings and media coverage on changes in antidepressant use, suicide attempts, and suicides among children/adolescents, young adults and adults. Evaluate the utility of sequential analysis for prospectively assessing signals of health policy impacts using FDA antidepressant warnings and related media coverage as policy example.
Description of study sample:
Records data from 11 MHRN health systems were used to examine time trends in rates of antidepressant use, suicide attempt, and suicide death before, during, and after FDA advisories regarding suicidality during antidepressant treatment.  The combined sample included approximately 1.1 million adolescents aged 10-17, 1.4 million adults aged 18-29, and 5 million adults aged 30-64.
Current Status: (write 1-2 sentences describing the project status; include current date)
Our latest publication in May 2018 evaluated the utility of sequential analysis for prospectively assessing signals of health policy impacts. As a policy example, we studied the consequences of the widely publicized Food and Drug Administration’s warnings cautioning that antidepressant use could increase suicidal risk in youth. Prospective, periodic evaluation of administrative health care data using sequential analysis can provide timely population-based signals of effects of health policies (see below). This method may be useful to use as new policies are introduced. Along with this publication Drs. Lu, Soumerai, Simon, and Kulldorff published point and counterpoint articles in Medical Care regarding the importance of surveillance (see below). Analysis for this project is complete and there will be no more publications. Using 28 years of US death certificate data collected and validated by the US CDC from 1990 to 2017, we are conducting the first longitudinal study of discontinuities in the trends of suicide rates before and after the warnings among adolescents and young adults. We hypothesized that the warnings and reductions in depression diagnosis and treatment would be associated with an increase in completed suicides among adolescents and young adults in the US. There are no extant national longitudinal data on the effects of this policy on completed suicides.
Study Registration:
N/A
Publications:Lu CY, Stewart C, Ahmed AT, Ahmedani BK, Coleman K, Copeland LA, Hunkeler EM, Lakoma MD, Madden JM, Penfold RB, Rusinak D, Zhang F, Soumerai SB. How complete are E-codes in commercial plan claims databases? Pharmacoepidemiol Drug Saf. 2014 Feb;23(2):218-20. doi: 10.1002/pds.3551.Lu CY, Zhang F, Lakoma MD, Madden JM, Rusinak D, Penfold RB, Simon G, Ahmedani BK, Clarke G, Hunkeler EM, Waitzfelder B, Owen-Smith A, Raebel MA, Rossom R, Coleman KJ, Copeland LA, Soumerai SB. Changes in antidepressant use by young people and suicidal behavior after FDA warnings and media coverage: quasi-experimental study. BMJ. 2014 Jun 18;348:g3596. doi: 10.1136/bmj.g3596.Lu, CY, Penfold RB, Toh S, Sturtevant J, Madden JM, Simon G, Ahmedani BK, Clarke G, Coleman KJ, Copeland L, Daida Y, Davis RL, Hunkeler EM, Owen-Smith A, Raebel MA, Rossom MA, Soumerai SB, Kulldorff M. Near real-time surveillance for consequences of health policies using sequential analysis. Med Care. 2018 May;56(5):365-372.Lu, CY, Simon, G, Soumerai, SB, Kulldorff, M. Early warning systems are imperfect, but essential.  Med Care. 2018 May;56(5):382-383.Lu, CY, Simon, G, Soumerai, SB. Staying honest when policy changes backfire. Med Care. 2018 May;56(5):384-390.
Resources:N/A
Lessons Learned: Completeness of e-codes varies significantly over time, across treatment settings and across study sites. Improvements in e-coding in commercial health plan datasets are critical for injury research. In the meantime, poisoning by psychotropic drugs appears to be a useful proxy for identifying suicide attempts leading to emergency room visits and hospitalizations. There were substantial reductions in antidepressant use among all age groups and simultaneous, small increases in psychotropic drug poisonings, a validated measure of suicide attempts, among adolescents and young adults following the FDA warnings. These results were consistent across 11 geographically diverse U.S. study sites. Media exaggeration about FDA reports of drug risks may reduce appropriate drug use and increase adverse outcomes. We did not detect changes in completed suicides after the warnings, which is an extremely rare outcome.
What’s Next?
The Virtual Data Warehouse (VDW) provides a rich resource for multi-site research.  The longitudinal nature of the VDW enables longitudinal analyses that are necessarily part of the interrupted time series method, a strong quasi-experimental study design for studying impacts of health policies. MHRN hosts a health policy special interest group for discussing these research ideas.

Practice Variation in High- and Low-value Care for Mood Disorders

Project Name:
Practice Variation in High- and Low-Value Care for Mood Disorders
Principal Investigator:
Gregory Simon MD MPH
Principal Investigator Contact Information:
simon.g@ghc.org
Principal Investigator institution:
Group Health Research Institute
Funder
NIMH
Funding Period:
09/2010 – 06/2015
Abstract:
This multi-site observational study examined patient, provider, and health system influences on process of depression care in primary care and mental health specialty settings.  Comprehensive records data from five MHRN sites (Group Health Cooperative, HealthPartners, Kaiser Permanente Colorado, Kaiser Permanente Hawaii, and Kaiser Permanente Southern California) were used to identify three patient cohorts: Primary care patients receiving a new diagnosis of depression with no recent history of depression treatment. Primary care and mental health specialty patients initiating a new episode of antidepressant treatment with a diagnosis of depression. Mental health specialty patients initiating a new episode of psychotherapy with a diagnosis of depression
Grant Number:
U19 MH092201 (Mental Health Research Network Cooperative Agreement)
Participating Sites:               
Group Health Cooperative                                                     
HealthPartners Institute                                
Kaiser Permanente Colorado                        
Kaiser Permanente Hawaii
Kaiser Permanente Southern California        
Investigators:
Gregory Simon MD MPH
Robert Penfold PhD
Susan Shortreed, PhD
Rebecca Rossom MD
Arne Beck PhD
Beth Waitzfelder PhD
Karen Coleman PhD
Major Goals:
To examine patient and provider contributions to variation in care (medication and psychotherapy) for depression.
Description of study sample:
The sample includes new diagnoses and new treatment episodes between 1/1/2010 and 12/31/2012.  These data are being used to address the following specific questions: Among primary care patients receiving a new diagnosis of depression, how do specific patient characteristics (age, sex, race/ethnicity, severity of depression) influence both the likelihood of initiating any treatment for depression and the choice between treatments (medication or psychotherapy)Among patients initiating medication treatment for depression, how are medication selection, early medication adherence, and acute-phase treatment response related to specific patient characteristics (age, sex, race/ethnicity, severity of depression)?  How do these treatment processes vary among providers? Among patients initiating psychotherapy for depression, how are early treatment adherence and acute-phase treatment response related to specific patient characteristics (age, sex, race/ethnicity, severity of depression)?  How do these treatment processes vary among providers?
Current Status:
All analyses are complete.
Study Registration:
N/A
Publications:
Simon GE, Coleman KJ, Waitzfelder BE, Beck A, Rossom RC, Stewart C, Penfold RB. Adjusting Antidepressant Quality Measures for Race and Ethnicity. JAMA Psychiatry. 2015 Oct;72(10):1055-6. doi: 10.1001/jamapsychiatry.2015.1437. No abstract available. PMID:26352783Simon GE, Rossom RC, Beck A, Waitzfelder BE, Coleman KJ, Stewart C, Operskalski B, Penfold RB, Shortreed SM.J. Antidepressants are not overprescribed for mild depression. Clin Psychiatry. 2015 Dec;76(12):1627-32. doi: 10.4088/JCP.14m09162.PMID:26580702Simon GE, Johnson E, Stewart C, Rossom RC, Beck A, Coleman KJ, Waitzfelder B, Penfold R, Operskalski BH, Shortreed SM.  Does patient adherence to antidepressant medication actually vary between physicians?  J Clin Psychiatry.  2017 Oct 24 (epub ahead of print)
Resources:
None
Lessons Learned: In MHRN health systems, we see little evidence for over-prescribing of antidepressants for mild depression. Likelihood of prematurely discontinuing antidepressant medication is much higher in minority racial and ethnic groups than in non-Hispanic Whites, and these racial and ethnic differences are far larger than differences related to other demographic or clinical characteristics. Likelihood of prematurely discontinuing psychotherapy for depression is modestly higher in minority racial and ethnic groups – but racial/ethnic disparities in psychotherapy adherence are smaller than disparities in antidepressant medication adherence. Among primary care patients receiving a new diagnosis of depression, likelihood of initiating any specific treatment (medication or psychotherapy) is lower among minority racial or ethnic groups.  Patients from minority racial and ethnic groups are more likely to start psychotherapy than medication. Failure to adjust antidepressant treatment quality measures for race and ethnicity will significantly disadvantage health systems serving members from traditionally under-served racial and ethnic groups. After accounting for random variation, likelihood of prematurely discontinuing antidepressant medication varies only minimally across physicians.
 What’s next?
A follow-up study (funded during the second cycle of MHRN funding) will further explore racial and ethnic disparities in care identified in this project.

Suicide Supplement: Development of a Population-Based Risk Calculator for Suicidal Behavior

Project Name:
Suicide Supplement: Development of a Population-Based Risk Calculator for Suicidal Behavior
Principal Investigator:
Greg Simon, MD MPH
Principal Investigator Contact Information:
Gregory.E.Simon@kp.org
Principal Investigator institution:
Kaiser Permanente Washington
Funder
NIMH
Funding Period:  
07/2015 – 06/2017
Abstract:
We propose to use population-based data from large health systems to develop evidence-based suicide attempt risk calculators for mental health and primary care clinicians.  Seven Mental Health Research Network (MHRN) sites will contribute data to this work.  Domains of predictors or risk indicators will include: Sociodemographic characteristics: age, sex, race/ethnicity, household socioeconomic statusGeneral clinical history: psychiatric diagnoses, co-occurring substance use disorder, co-occurring medical illness, outpatient treatment history, inpatient treatment historySuicidal behavior history: prior suicide attempt, other prior injury or poisoningSuicidal ideation history: number, timing, and results of previous responses to PHQ item 9Current presentation: depression and anxiety symptom severity, frequency/intensity of suicidal ideation, current substance useNonfatal and fatal suicide attempts will be identified using health system records and state mortality data.  Analyses will estimate cumulative hazard of suicide attempt over 30, 90, and 180 days following an index encounter, contingent on specific characteristics in each of the five predictor domains listed above.  We will build predictive models using all observations on the same individual over time as well as randomly sampling one observation per individual – to assess the bias in the risk prediction model in the combined population.  We will use statistical learning methods to build and evaluate prediction models to identify who is at increased risk of suicide and when that risk is reduced or elevated.  Results of these analyses will inform creation of EHR-based risk calculator tools to support outpatient providers’ decisions regarding suicide risk assessment and follow-up care.  Distinct models and decision support tools will be used to inform pre-visit planning (using all risk factor information present prior to the index visit) and within-visit planning (using additional information recorded during the index visit). 
Grant Number:  
U19MH092201 (Supplement under MHRN II)
Participating Sites:               
Kaiser Permanente, Washington
Henry Ford Health Systems, Michigan
HealthPartners Institute for Education and Research, Minnesota
Kaiser Permanente, Colorado
Kaiser Permanente, Hawaii
Kaiser Permanente, Northwest
Kaiser Permanente, Southern California
Investigators:
Gregory Simon, MD, MPH
Brian Ahmedani, PhD
Rebecca Rossom, MD, MSCR
Arne Beck, PhD
Beth Waitzfelder, PhD
Frances Lynch, PhD
Karen Coleman, PhD
Major Goals:
This study will inform creation of EHR-based risk calculator tools to support outpatient providers’ decisions regarding suicide risk assessment and follow-up care.  Our consultations with stakeholders (both front-line clinicians and health system leaders) identify two key information needs: Pre-visit planning – Prior to each visit (all mental health specialty visits or primary care visits for patients with mental health conditions), treating providers would receive a risk prediction based on clinical information available prior to the appointment. This predicted risk score would be calculated using Epic’s Reporting Workbench functions and displayed in each provider’s Epic Schedule Review function, following a process now used for other clinical risk prediction tools in participating health systemsWithin-visit assessment – During each visit, entry of PHQ9, GAD2/7 or AUDIT-C questionnaire data would trigger an updated risk prediction, incorporating most recent response to PHQ item 9 as well as other symptom severity scales. This predicted risk score would be calculated and displayed using Epic’s SmartLink function, following a process now used for cardiovascular risk predictions. In addition to serving different practical needs (pre-visit preparation and within-visit treatment planning), these alternative models will address a more general scientific or public health question: the relative importance of long-term characteristics and time-varying or immediate characteristics in predicting risk of suicidal behavior.
Description of study sample:
The sample will include all patients aged 13 or older with at least one outpatient visit between 1/1/2009 and 6/30/2015 that is either with a specialty mental health provider OR with a recorded diagnosis of mood, anxiety, personality, or psychotic disorder. Sample includes 19.6 million visits for approximately 2.9 million people.
Current Status:
Data have been collected from participating sites and combined, creating one analytic dataset.  Primary analyses are complete (as of 3/1/2018) with some secondary analyses ongoing.
Study Registration:
N/A
Publications:
Simon GE, Johnson E, Lawrence JM, Rossom RC, Ahmedani B, Lynch FL, Beck A, Waitzfelder B, Ziebell R, Penfold RB, Shortreed SM.  Predicting suicide attempts and suicide deaths following outpatient visits using electronic health records.  Am J Psychiatry 2018: May 24 (epub ahead of print)Simon GE, Shortreed SM, Coley RY, Penfold RB, Rossom RC, Waitzfelder BE, Sanchez K, Lynch FL. Assessing and Minimizing Re-identification Risk in Research Data Derived from Health Care Records. EGEMS (Wash DC). 2019 Mar 29;7(1):6. doi: 10.5334/egems.270.Simon GE, Yarborough BJ, Rossom RC, Lawrence JM, Lynch FL, Waitzfelder BE, Ahmedani BK, Shortreed SM. Self-Reported Suicidal Ideation as a Predictor of Suicidal Behavior Among Outpatients With Diagnoses of Psychotic Disorders. Psychiatr Serv. 2019 Mar 1;70(3):176-183. doi: 10.1176/appi.ps.201800381. Epub 2018 Dec 10.
Resources:
Code for identifying the study sample and computing predictors is available via the MHRN GitHub site:  https://github.com/MHResearchNetwork
Lessons Learned: Prediction models using electronic health records data can accurately identify outpatients at increased risk for suicide attempt and suicide death. Separate prediction models for adolescents are not necessary. Self-reported suicidal ideation accurately predicts suicide attempt in people with psychotic disorders.
What’s next? Secondary analyses may lead to additional manuscripts regarding: Risk prediction in adolescentsRisk prediction in people with substance use disordersDisagreement between risk models and self-reported suicidal ideationVariation in accuracy of risk prediction across racial and ethnic groups

Treatment Utilization Before Suicide

Project Name:
Treatment Utilization Before Suicide
Principal Investigator:
Brian Ahmedani, PhD
Principal Investigator Contact Information:
BAHMEDA1@HFHS.ORG
Principal Investigator institution:
Henry Ford Health System Research Centers
Funder:
NIMH
Funding Period:
03/2015 – 02/2020
Adult suicide rates in the United States rose by almost 30 percent between 1999 and 2010. These rates have not markedly improved in decades. To date, previous suicide attempts and psychiatric diagnoses are largely the only known clinical risk factors for suicide death. Recent research shows that most individuals who die by suicide make a health care visit in the weeks and months prior to their death. Most of these visits occur in primary care or outpatient medical specialty settings. However, over half of these visits do not include a psychiatric diagnosis. Thus, there is limited evidence available from health care users in the US general population to inform targeted suicide screening and risk identification efforts in general medical settings. New research is needed to investigate the general medical clinical factors associated with suicide risk among individuals without a known risk factor. This research project uses data on more than 4000 individuals who died by suicide and made health care visits to one of eight health care systems across the United States in the year prior to their death. These health systems are members of the Mental Health Research Network and have affiliated health plans. They are able to capture nearly all health care for their patients via the Virtual Data Warehouse (VDW). The VDW consists of electronic medical record and insurance claims data organized using standardized data structures and definitions across sites. These data are matched with official regional mortality data. This project includes the following Specific Aims: 1) Identify the types and timing of clinical factors prior to suicide, 2a) Compare clinical factors before suicide to a matched sample of health care users, 2b) Detect associations between additional clinical factors and suicide, and 3) Develop a prediction model of clinical factors prior to suicide. We employ a case-control study approach to test specific hypotheses, while also using novel environment-wide association study methods and latent class analysis to detect new risk factors. We develop a prediction model of clinical factors and suicide. Clinical factors to be studied include medical diagnoses, medications, health care procedures, and types of health care visits. These results will inform decisions about how to focus suicide prevention in medical settings and provide information in response to the 2012 National Action Alliance for Suicide Prevention and US Surgeon General report.
Grant Number:
R01MH103539
Participating Sites:       Henry Ford Health System
Harvard Pilgrim Healthcare
HealthPartners
Kaiser Permanente Hawaii
Kaiser Permanente Northwest
Kaiser Permanente Colorado
Kaiser Permanente Georgia
Kaiser Permanente Washington
Investigators:
Brian K. Ahmedani, PhD
Gregory E. Simon, MD, MPH
Rebecca Rossom, MD, MSCR
Arne Beck, PhD
Frances Lynch, PhD
Beth Waitzfelder, PhD
Christine Lu, PhD
Ashli Owen-Smith, PhD
Deepak Prabhakar, MD, MPH
L. Keoki Williams, MD, MPH
Edward Peterson, PhD
Cathrine Frank, MD
Major Goals:
The main goal of this project is to investigate general medical and other healthcare factors and risk of suicide to develop a comprehensive healthcare algorithm to predict suicide, with particular focus on general medical settings.
Description of study sample:
This large case-control study includes >3,000 individuals who died by suicide between 2000-2015 and >300,000 matched general population members of 8 large health systems across the United States. 
Current Status:
June 26, 2019: Aims 1-2 have been completed. The work in Aim 3 is currently underway, including developing a series of predictive models for the full sample and a series of subgroups..  We will complete data analysis and draft the manuscript in Winter 2019-2020.
Study Registration:
N/A
Publications:Ahmedani BK, Simon GE, Stewart C, Beck A, Waitzfelder BE, Rossom R, Lynch F, Owen-Smith A, Hunkeler EM, Whiteside U, Operskalski BH, Coffey MJ, Solberg LI. Health care contacts in the year before suicide death. J Gen Intern Med. 2014 Jun;29(6):870-7. doi: 10.1007/s11606-014-2767-3. PMID: 24567199Ahmedani BK, Stewart C, Simon GE, Lynch F, Lu CY, Waitzfelder BE, Solberg LI, Owen-Smith AA, Beck A, Copeland LA, Hunkeler EM, Rossom RC, Williams K. Racial/Ethnic differences in health care visits made before suicide attempt across the United States. Med Care. 2015 May;53(5):430-5. doi: 10.1097/MLR.0000000000000335. PMID: 25872151.Ahmedani BK, Peterson EL, Hu Y, Rossom RC, Lynch F, Lu CY, Waitzfelder BE, Owen-Smith AA, Hubley S, Prabhakar D, Williams LK, Zeld N, Mutter E, Beck A, Tolsma D, Simon GE. Major Physical Health Conditions and Risk of Suicide. Am J Prev Med. 2017 Sep;53(3):308-315. doi: 10.1016/j.amepre.2017.04.001. PMID: 28619532.Boggs JM, Simon GE, Ahmedani BK, Peterson E, Hubley S, Beck A. The Association of Firearm Suicide With Mental Illness, Substance Use Conditions, and Previous Suicide Attempts. Ann Intern Med. 2017 Aug 15;167(4):287-288. doi: 10.7326/L17-0111. PMID: 28672343.Prabhakar D, Peterson EL, Hu Y, Rossom RC, Lynch FL, Lu CY, Waitzfelder BE, Owen-Smith AA, Williams LK, Beck A, Simon GE, Ahmedani BK. Dermatologic Conditions and Risk of Suicide: A Case-Control Study. Psychosomatics. 2018; 59(1): 58-61. doi: 10.1016/j.psym.2017.08.001. PMID: 28890116.Boggs JM, Beck A, Hubley S, Peterson EL, Hu Y, Williams LK, Prabhakar D, Rossom RC, Lynch FL, Lu CY, Waitzfelder BE, Owen-Smith AA, Simon GE, Ahmedani BK.General Medical, Mental Health, and Demographic Risk Factors Associated With Suicide by Firearm Compared With Other Means. Psychiatric Services. 2018; 69(6):677-684. doi: 10.1176/appi.ps.201700237. PMID: 29446332.Owen-Smith AA, Ahmedani BK, Peterson E, Simon GE, Rossom RC, Lynch FL, Lu CY, Waitzfelder BE, Beck A, DeBar LL, Sanon V, Maaz Y, Khan S, Miller-Matero LR, Prabhakar D, Frank C, Drake CL, Braciszewski JM. The Mediating Effect of Sleep Disturbance on the Relationship Between Nonmalignant Chronic Pain and Suicide Death.  Pain Pract. 2019 Apr;19(4):382-389. doi: 10.1111/papr.12750. Epub 2019 Jan 18. PMID: 30462885Yeh HH, Westphal J, Hu Y, Peterson EL, Williams LK, Prabhakar D, Frank C, Autio K, Elsiss F, Simon GE, Beck A, Lynch FL, Rossom RC, Lu CY, Owen-Smith AA, Waitzfelder BE, Ahmedani BK. Diagnosed Mental Health Conditions and Risk of Suicide Mortality. Psychiatr Serv. 2019 Jun 12:appips201800346. doi: 10.1176/appi.ps.201800346. [Epub ahead of print]. PMID: 31185853
Resources:
None
Lessons Learned: Most individuals make healthcare visits before suicide. Most visits occur in primary care or general medical specialty settings. Approximately half of individuals do not have a mental health condition diagnosed during their health care visits before suicide. Among 19 physical health conditions under study, 17 were associated with increased risk for suicide after adjustment for age and sex, and 9 associations persisted after additional adjustment for mental health and substance use conditions.
What’s next?
The final predictive modeling analyses are underway for the final study aim. A series of papers are currently under review or in development based on data from Aims 1-2.