Tag: Analytic methods

Optimizing Care to Prevent Diabetes and Promote Cardiovascular Health Among Younger Adults with Severe Mental Illness

Grant Details

Funder: NIMH

Grant number: 1K23MH126078

Grant period: 4/1/2022 – 3/31/2027

Brief narrative: People with severe mental illness (SMI) face double the risk for type 2 diabetes compared to the general population, contributing to higher rates of cardiovascular disease and premature death. Common use of antipsychotic medications contributes to these health risks due to prevalent metabolic side effects. Many younger adults with SMI do not receive targeted, evidence-based cardiometabolic disease prevention care. Underused strategies include: prescribing alternative, less obesogenic psychotropic medications; lifestyle change supports; additional risk-reducing medications; and smoking cessation therapies. Our preliminary qualitative data with patients and clinicians identified a need for tools to match prevention care to individuals’ risk level and preferences, and tools suited to population-based care strategies. Clinical Decision Support (CDS) tools are computer algorithms that use patients’ data, predictive analytics, and clinical guidelines to promote evidence-based care by helping patients and clinicians navigate complex treatment decisions. Through this mentored K23 career development award, Esti Iturralde, PhD will build upon her background as a clinical psychologist and behavioral diabetes researcher. Through planned mentoring, coursework, and career development activities, Dr. Iturralde will gain a strong understanding of psychopharmacology and cardiometabolic health, advanced predictive analytics, and implementation science, including methods for stakeholder-engaged intervention design and pragmatic clinical trials. As a researcher in the Kaiser Permanente Northern California (KPNC) Division of Research (DOR), she will leverage robust, longitudinal electronic health record (EHR) data (> 50,000 adults from diverse racial/ethnic groups) and stakeholder insights (patients, clinicians, and health system decision-makers) within health systems including KPNC and 2 others belonging to the NIMH-funded Mental Health Research Network (HealthPartners Institute and Henry Ford Health System). The proposed research will support the training goals while contributing to the development of a novel CDS tool seeking to increase targeted, evidence-based diabetes and cardiovascular disease prevention care for adults under age 45 who are starting antipsychotic medications. Specific research aims are to: (1) inform predictive analytics of the CDS tool by developing and validating diabetes risk prediction models for the target population; (2) engage stakeholders in the design of CDS tool messaging and implementation pathways; and field-test CDS tool messaging through a pragmatic clinical trial conducted within an existing KPNC telehealth-based population management program serving this population. A future R01 application will build on the results from this project to further refine and test the CDS tool within multiple health systems. The linked research and training aims will directly prepare Dr. Iturralde for success as an embedded health system researcher and prepare her to lead a programmatic line of studies developing and implementing data-driven, feasible, scalable interventions improving the cardiometabolic health of people with SMI.

Lead site: KPNC (PI Esti Iturralde)

Current Status

Summary of Findings

Publications

Improving Suicide Risk Prediction with Social Determinants Data

Grant Details

Funder: NIMH

Grant Number: R56MH125794-01A1

Grant Period: 1/1/2022 – 12/31/2022

Brief Narrative: Suicide accounted for 47,511 deaths in the United States in 2019 and the suicide rate has increased by 39% since 1999. Suicide prevention is an NIMH research priority. Recent research in estimating machine learning algorithms to predict suicide risk has been tremendously successful. The models have been implemented as part of routine prevention programs in health systems such as Kaiser Permanente Washington, HealthPartners, and the Veterans Health Administration. Despite these successes, existing models have important shortcomings. A significant proportion of suicides followed healthcare visits where the predicted risk was low (and where an intervention might have taken place otherwise). The models do not currently include any information about social determinants of suicide (e.g., living alone, financial stress) or negative life events (NLE), such as divorce, bankruptcy, and criminal arrest. Adding social determinants data and NLE data to models may improve predictive accuracy. The specific aims of this study are: (1) expand and enhance the risk prediction dataset with over 1500 date-stamped variables describing social determinants of suicide risk and NLE; (2) construct and evaluate suicide risk prediction models using social determinants and NLE data alone; (3) construct and evaluate suicide risk prediction models using social determinants, NLE and healthcare data together and estimate interaction terms between social determinants, NLE, and healthcare predictors. An example would be “depression diagnosis” interacted with “divorce filing in last 30 days”. This will be the first large scale study to incorporate individual-level, date-stamped measures of social determinants and NLE into machine learning suicide risk prediction models. Upon successful completion of this study we expect to know how much incorporating these new data contributes to the accuracy of suicide risk prediction models. This will be an important next step towards implementing better suicide prevention programs and reducing overall suicide rates.

Lead Site: KPWA (PI Rob Penfold)

Participating Sites: N/A

Current Status

We fielded the discrete choice experiment in mid-October 2022. Planned recruitment is 720.

Summary of Findings

Publications

Trans-America Consortium of the Health Care Systems Research Network for the All of Us Research Program

Grant Details

Funder: NIH Office of the Director

Grant Number: OT2OD026550

Grant Period: 1/4/2018 – 3/31/2023

Narrative:

Lead Site: HFHS (co-PIs Christine Johnson and Brian Ahmedani)

Participating Sites:

Current Status:

Ongoing recruitment, enrollment and retention of 100,000 participants and members.

Summary of Findings:

Publications:

Cronin, R.M., Jerome, R.N., Mapes, B.M., Andrade, R., Johnston, R., Ayala, J., Schlundt, D., Bonnet, K.R., Kripalani, S., Goggins, K., Wallston, K.A., Couper, M.P., Elliott, M.R., Harris, P.A., Begale, M.A., Munoz, F.A., Lopez-Class, M., Cella, D., Condon, D.M., AuYoung, M., Mazor, K.M., Mikita, S., Manganiello, M., Borselli, N., Fowler, S.L., Rutter, J.L., Denny, J.C., Karlson, E.W., Ahmedani, B.K., O’Donnell, C.J. Vanderbilt University Medical Center Pilot Team, and the Participant Provided Information Committee. (2019). Development of the Initial Surveys for the All of Us Research Program. Epidemiology, 30(4), 597-608.. doi: 10.1097/EDE.0000000000001028. PMID: 31045611. 

Ramirez AH, Sulieman L, Schlueter DJ, Halvorson A, Qian J, Ratsimbazafy F, Loperena R, Mayo K, Basford M, Deflaux N, Muthuraman KN, Natarajan K, Kho A, Xu H, Wilkins C, Anton-Culver H, Boerwinkle E, Cicek M, Clark CR, Cohn E, Ohno-Machado L, Schully SD, Ahmedani BK, Argos M, Cronin RM, O’Donnell C, Fouad M, Goldstein DB, Greenland P, Hebbring SJ, Karlson EW, Khatri P, Korf B, Smoller JW, Sodeke S, Wilbanks J, Hentges J, Mockrin S, Lunt C, Devaney SA, Gebo K, Denny JC, Carroll RJ, Glazer D, Harris PA, Hripcsak G, Philippakis A, Roden DM; All of Us Research Program. (2022). The All of Us Research Program: Data quality, utility, and diversity. Patterns (N Y); 3(8), 100570. doi: 10.1016/j.patter.2022.100570. PMID: 36033590.

Cronin, R.M., Halvorson, A.E., Springer, C., Feng, X., Sulieman, L., Loperena-Cortes, R., Mayo, K., Carroll, R.J., Chen, Q., Ahmedani, B.K., Karnes, J., Korf, B., O’Donnell, C.J., Qian, J., Ramirez, A.H., All of Us Research Program Investigators.  (2021). Comparison of Family Health History in Surveys versus Electronic Health Records in the All of Us Research Program. Journal of the American Medical Informatics Association, 28(4):695-703. doi: 10.1093/jamia/ocaa315. PMID: 33404595. 

Predictive modeling: the role of opioid use in suicide risk

Grant Details

Funder: NIDA

Grant Number: R01DA047724

Grant Period: 8/15/2018 – 6/30/2022

Narrative: Suicide deaths and opioid-related overdose deaths have both been increasing in recent years. These two public health crises have substantial overlap: our preliminary work suggests that between 22% and 37% of opioid-related overdoses are suicides or suicide attempts. Healthcare settings are ideal places to intervene to prevent suicides, however clinicians need better tools to recognize the patients at greatest risk. We developed models that predict risk of suicide attempt or death with 83% to 86% accuracy. However, these models do not include important opioid-related variables. In a parallel body of work, we developed algorithms based on coded electronic health record (EHR) data to identify opioid-related overdoses and classify them as unintentional or intentional suicides. The proposed project integrates these two existing lines of research. Our suicide risk prediction dataset includes seven large healthcare systems and approximately 20 million visits by 3 million patients; it is currently being expanded to include additional outcomes and visits through 2016, and additional predictors, however inclusion of opioid-related variables was not part of the funded supplement. In the proposed study, we will determine whether including variables related to illicit and prescribed opioid use, opioid use disorder, discontinuation or significant dose reductions of prescription opioids, or prior non-fatal opioid-related overdoses improves predictions of suicide attempts or death within 90 days following an outpatient healthcare visit. We will also develop models that specifically predict opioid- related suicide attempts and deaths in the sample as a whole and among people prescribed opioid medications, and determine if the predictors of opioid-related suicide attempts or deaths are consistent for men and women. The goal of the proposed work is to maximize the performance of our models in order to create the best available tools for clinicians to help reduce future suicides. We have an established collaboration with the largest national EHR vendor and are working to develop an EHR-based, point-of-care clinical tool to predict suicide attempts and deaths based on our research findings. This work will therefore have a direct impact on clinical practice by providing clinicians with an efficient, evidence-based tool to evaluate suicide risk. The work will also provide critical data on understudied opioid-related predictors and moderators of suicide.

Lead Site: KPNW (PI Bobbi Jo Yarborough)

Participating Sites: HFHS, HPI, KPCO, KPHI, KPSC, KPWA

Current Status

Summary of Findings

Publications

Assisted Identification and Navigation of Early Mental Health Symptoms in Youth

Grant Details

Funder: NIMH

Grant Number: R01MH124652

Grant Period: 1/18/2021 – 11/30/2024

Narrative: About 55% of children with significant mental health difficulties receive treatment and up to 80% of children with sub-clinical symptoms receive no treatment. Treatments are often not initiated until issues are significantly impacting the child and family. This study aims to conduct a pragmatic randomized trial in two non-academic health care systems to test a mental health family navigator model to promote early access to, engagement in, and coordination of needed mental health services for children. The first task of the study will focus on the implementation of a predictive model to identify symptomatic children with no diagnosed mental health disorder(s) or treatments initiated. The tool identifies patients with documentation of mental health symptoms or complaints in the free text of a progress note from a recent primary care or urgent care visit. Using this predictive algorithm, we will conduct a pragmatic randomized trial comparing intervention and usual care arm patients enrolled from Kaiser Permanente (KP) Washington and KP Northern California. The trial will enroll 200 patients per arm (n=400). Children with (1) a new mental health diagnosis but no treatment initiated; (2) a new mental health medication ordered with no mental health diagnosis; and (3) symptoms identified by the predictive model with no new mental health diagnosis or treatment initiated will be recruited. The study intervention will offer 6 months of support to the family by a mental health navigator (social worker). The navigator will perform an initial needs and barriers assessment with the family around mental health services, conduct ongoing motivational interviewing around mental health care, provide up to 4 psychotherapy sessions (when appropriate) via clinic-to-home video visits, help the family find and schedule with appropriate mental health providers in the community, and reach out ad hoc if mental health appointments or medication refills are missed. The primary outcome is the percentage of youth initiating psychotherapy. The secondary outcome is the percentage of youth with at least 4 mental health visits. We hypothesize that the intervention arm will have higher rates of psychotherapy use compared to the control arm. We will also assess initiation of psychotropic medications. All primary analyses will follow an intent-to-treat approach. A waiver of consent will be obtained to include data for all individuals offered the intervention in the analysis, regardless of the amount of intervention (“dose” of navigation) received.

Lead Site: KPWA (PI Rob Penfold)

Participating Site: KPNC

Current Status:

Recruitment is active at both KPWA and KPNC. N = 44 as of 10/25/2022.

Summary of Findings:

Publications:

Using NLP to Increase Identification of Child Maltreatment in EHR

Grant Details

Funder: NIMH (MHRN III Feasibility Pilot Program)

Grant Number: U19MH121738

Project Period: 7/1/2020 – 6/30/2021

Narrative:

Background: Child maltreatment is a critical public health issue and health care systems play an important role in identifying and treating children who experience maltreatment. To date, few studies of child maltreatment have used data from large health systems to try and understand how these systems identify and manage youth who experience maltreatment. Preliminary analyses of the number of children identified as having experienced child maltreatment in the most recent MHRN quarterly descriptive analyses (2018) indicate that there is likely a significant under-reporting of child maltreatment in the MHRN health systems. Epidemiologic studies suggest that many more youth would have been identified with child maltreatment. One reason for this potential under reporting is that providers may not use the ICD codes to document child maltreatment consistently. Some maltreatment may be discussed in chart notes but not documented using ICD codes. Better identification of maltreatment could aid both research and practice within health care systems. Natural Language Processing may help to identify additional youth with maltreatment. If NLP identifies cases that are not documented through ICD codes, this could indicate the need for health system efforts to develop new ways of consistently document child maltreatment. NLP might also help to identify any groups (e.g., age, gender, race/ethnicity) that may be particularly likely to have insufficient documentation of child maltreatment. 

This work aligns with NIMH’s strategies to increase research and improve outcomes of mental health services in diverse and vulnerable populations, and to conduct research that helps health systems to base care decisions on the best possible data.   

Research Question: The overarching question is does NLP allow us to obtain estimates the number of children who experience maltreatment more comparable to national epidemiologic data? Does NLP of chart notes identify new cases of child maltreatment that are not already documented with ICD codes? What is the overlap between the two methods? Are there differences by age group or race-ethnicity? Does NLP allow us to differentiate between new/current maltreatment versus history of maltreatment?

Methods: We propose to use simple NLP queries at 1 MHRN site (e.g., terms such as physical abuse, maltreatment) to search chart notes and to compare the number of cases identified through NLP and compare those to cases identified through ICD codes. We will also conduct analyses to see if there is variation in identification by age group, gender, and race/ethnicity.  

Planned Product: We plan to write a paper documenting our findings. We also plan to write a grant related to child maltreatment using this data.

  • Lead Site: KPWA (PI Rob Penfold)
  • Participating Sites:
  • KPSC (Co-I Sonya Negriff)
  • KPNW (Co-I Frances Lynch)

Current Status

  • NLP pipeline created
  • Manual adjudication of NLP “hits” complete
  • Descriptive statistics complete

Summary of Findings

The prevalence of child maltreatment as measured by adjudicated occurrences of terms and phrases discovered by NLP is much higher than when measured via discrete data elements.

Publications

None yet

Improved tailoring of depression care using customized clinical decision support

Grant Details

Funder: NIMH

Grant number: R01MH114873

Grant period: 07/01/2018 – 04/30/2023

Narrative: Treatments for mental health conditions such as unipolar depression provide modest average benefit but have wide variation between individuals and within individuals over time. Evidence-based customized treatment protocols would improve the mental health care of many people by providing treatment recommendations for individuals that take into account potential variation because of personal characteristics such as current health status, symptoms, and response to earlier treatment. Generating customized treatment protocols requires large amounts of data, such as from networks of health systems that can link electronic health records from millions of individuals. Current statistical approaches for discovering customized treatment protocols are limited in three important ways. First, current approaches rely on scientists to select the patient characteristics to use to customize treatments instead of using data to find the patient characteristics that will lead to improved, customized care. Second, customized treatment protocols discovered with current statistical methods assume no unobserved differences between individuals who receive various treatment options. Third, investigators do not have ways to know if the available data contain enough information to discover and compare customized treatment protocols precisely enough to make clinical decisions. We will address these three limitations by developing new statistical tools for discovering customized treatment protocols using electronic health records data. Our research team has expertise and experience in statistics, epidemiology, and mental health care. We will integrate methods that have been successfully used in other settings to improve statistical approaches for discovering customized treatment protocols and address these three important limitations. We will extend machine learning tools for selecting important pieces of information to the time-varying data structure required for discovering customized treatment protocols. We will build approaches that use available knowledge about the size of unobserved differences between groups of people who received different treatments to assess how those differences change study results. By building on the math used to estimate the sample sizes needed for precision in randomized trials with complex designs, we will develop new formulas for determining how many people with a particular condition and who took a particular drug are needed in a health system to provide enough accurate information to discover customized treatment protocols. Using data from the electronic health records of more than 15,000 patients, we will discover customized treatment protocols for depression. By improving statistical tools and addressing current limitations, our customized treatment protocols will have immediate impact for people living with unipolar depression. The statistical tools we develop will also be useful for discovering customized treatment protocols for people with a wide variety of mental health conditions.

Lead site: KPWA (PI Susan Shortreed)

Participating site: McGill University (co-I Erica Moodie)

  • Funder contacts:
    • Program Official: Michael Freed

Current Status

We have published papers proposing approaches to sample size estimation, unmeasured confounding sensitivity analyses, and selecting tailoring variables. We are continuing to work on alternative methods for tailoring variable selection.

Summary of Findings

  • Shrinkage regression based methods can identify important tailoring variables
  • Distributed regression methods can optimize individual treatment rules while protecting individual privacy
  • Dynamic weighted survival modeling can identify more effective individualized antidepressant treatment strategies using health records data

Publications

  1. Coulombe J, Moodie EEM, Shortreed SM, Renoux C. Can the Risk of Severe Depression-Related Outcomes Be Reduced by Tailoring the Antidepressant Therapy to Patient Characteristics? Am J Epidemiol. 2021 Jul 1;190(7):1210-1219. doi: 10.1093/aje/kwaa260. PMID: 33295950; PMCID: PMC8245894.
  2. Bian Z, Moodie EEM, Shortreed SM, Bhatnagar S. Variable selection in regression-based estimation of dynamic treatment regimes. Biometrics. 2021 Nov 27. doi: 10.1111/biom.13608. Epub ahead of print. PMID: 34837380.
  3. Moodie EEM, Coulombe J, Danieli C, Renoux C, Shortreed SM. Privacy-preserving estimation of an optimal individualized treatment rule: a case study in maximizing time to severe depression-related outcomes. Lifetime Data Anal. 2022 Jul;28(3):512-542. doi: 10.1007/s10985-022-09554-8. Epub 2022 May 2. PMID: 35499604.

Reduce Racial/Ethnic Disparities in Suicide Risk Prediction (RED)

Grant Details

Title: Innovative methods to reduce racial and ethnic disparities in suicide risk prediction

Funder: NIMH

Grant number: 1R01MH125821

Grant period: 1/1/2022 – 12/31/2025

Brief Narrative: Suicide risk prediction models are being used by health care systems to guide delivery of suicide prevention interventions, but these prediction models may not accurately identify high-risk patients in racial and ethnic subgroups that are less prevalent or have lower rates of suicide attempt and death. This project will reduce racial and ethnic disparities in suicide risk models by developing methods for prediction model estimation that optimize performance within subgroups, rather than across the whole population, and adjust for misclassification of suicide outcomes. We will also design sample size calculations that evaluate the ability of a prediction study to accurately identify high-risk individuals within racial and ethnic subgroups.

  • Lead site:
    • KPWA (PI Yates Coley)
  • Participating sites:
    • University of Washington (Co-I Noah Simon)
    • KPSC (Co-I Karen Coleman)

Awarded budget (total cost): $1,622,626

Human Subjects: Reviewed by KPWA IRB, IRBNet# 1870253

Current status

Statistical methods research is underway. IRB and data use approvals are in place for all planned analyses. Current activities are focused on methods for accounting for outcome misclassification; evaluating variable importance in suicide prediction models; and designing estimation methods to optimize performance in racial/ethnic subgroups.

Summary of findings

Publications

MHRN Post-Doctoral Fellowship Program

About

The MHRN T32 Post-Doctoral Fellowship is an NIMH-supported, two-year fellowship program. Aspiring independent researchers are trained broadly in health systems/health services research with a focus on mental health conditions and services, as well as suicide prevention. Areas of particular interest and training are in clinical interventions, healthcare service delivery, big data, implementation science, comorbidities, health equity, and health policy. The overall goal of the program is to support fellows in the transition to becoming independent mental health services researchers able to pursue NIMH-funding to support research within health system settings.

Fellows work locally alongside a primary mentor at HFHS or KPNC. They also receive high-quality mentorship and training from other scientists within multiple formats across the entire MHRN. In addition to one-on-one mentoring in the fellows’ areas of interest, they receive training in grant writing, manuscript development, health systems research methods (e.g., case-control designs, big data science, multi-site trials, dissemination and implementation), quantitative and qualitative methods, ethics, professional/career development, and conducting clinical trials. Teaching opportunities may also be available, as will clinical experiences for those seeking licensure.

Two fellows are enrolled each year. Applicants should have a PhD, MD, or other doctoral degree in a related field. Open to U.S. citizens or permanent residents enrolled in research or clinical doctoral or postdoctoral programs.

The Program Lead is Brian Ahmedani (HFHS). The Training Directors are Stacy Sterling (KPNC) and Jordan Braciszewski (HFHS).

A full description of the program and the application process can be viewed at https://www.henryford.com/hcp/research/public-population-research/health-policy/research .

Current Fellows

Seminar Schedule

seminar materials

Contact

For more information about the MHRN Fellowship program, contact MHRNT32@hfhs.org.

MHRN III Infrastructure: Methods Core

Grant Details

Funder: NIMH

Grant Number: U19MH121738

Grant Period: 09/23/2019 – 06/30/2020

Narrative: The Methods Core will include an Informatics Unit, led by Drs. Gregory Simon and Christine Stewart, and a Scientific Analysis Unit, led by Drs. Susan Shortreed and Patrick Heagerty. The Informatics Unit will continue highly successful work over the past 8 years, supporting routine data quality assessment and descriptive analyses of diagnosis and treatment patterns across all participating health systems. New work will include development of tools and resources to assess and minimize privacy risks when sharing sensitive health data for research and development of specific new data areas (perinatal mental health and prenatal exposures, expanded list of patient-reported outcomes, and assessments of social determinants of health). The Informatics Unit will provide consultation to all MHRN core and affiliated projects and share all resources with other researchers and health systems via MHRN’s public repository of specifications, code lists, and analytic code. The Scientific Analysis Unit will support to all MHRN core and affiliated projects via project-specific consultation and development of a learning community of analysts and biostatisticians across MHRN research centers. This Unit will also focus on development and dissemination of analytic methods in two areas directly relevant to MHRN research. Work on evaluating adaptive treatment strategies will build on Dr. Shortreed’s recently funded methods grant to evaluate and disseminate methods for using health system data to tailor treatments for individuals with more chronic or severe mental health conditions, focusing on assessing treatment effects when treatments are adjusted or switched according to previous treatment failures or adverse effects. Work on stakeholder-driven predictive analytics will build on MHRN’s development of accurate suicide risk prediction models, focusing on matching specific study designs and model development methods with stakeholder priorities and implementation constraints.

Lead Site: KPWA (PI Greg Simon)

Participating Sites: University of Washington (Site PI Patrick Heagerty) 

  • Funder Contacts
    • Science Officer: Susan Azrin
    • Program Official: Michael Freed
    • Grants Management Official: Jackie Chia

Documents & Reports

Submitted Proposal

Specific Aims

Research Plan

Notice of Award

Personnel Contact List

Publications