Trans-America Consortium of the Health Care Systems Research Network for the All of Us Research Program

Grant Details

Funder: NIH Office of the Director

Grant Number: OT2OD026550

Grant Period: 1/4/2018 – 3/31/2023

Narrative:

Lead Site: HFHS (co-PIs Christine Johnson and Brian Ahmedani)

Participating Sites:

Current Status:

Ongoing recruitment, enrollment and retention of 100,000 participants and members.

Summary of Findings:

Publications:

Cronin, R.M., Jerome, R.N., Mapes, B.M., Andrade, R., Johnston, R., Ayala, J., Schlundt, D., Bonnet, K.R., Kripalani, S., Goggins, K., Wallston, K.A., Couper, M.P., Elliott, M.R., Harris, P.A., Begale, M.A., Munoz, F.A., Lopez-Class, M., Cella, D., Condon, D.M., AuYoung, M., Mazor, K.M., Mikita, S., Manganiello, M., Borselli, N., Fowler, S.L., Rutter, J.L., Denny, J.C., Karlson, E.W., Ahmedani, B.K., O’Donnell, C.J. Vanderbilt University Medical Center Pilot Team, and the Participant Provided Information Committee. (2019). Development of the Initial Surveys for the All of Us Research Program. Epidemiology, 30(4), 597-608.. doi: 10.1097/EDE.0000000000001028. PMID: 31045611. 

Ramirez AH, Sulieman L, Schlueter DJ, Halvorson A, Qian J, Ratsimbazafy F, Loperena R, Mayo K, Basford M, Deflaux N, Muthuraman KN, Natarajan K, Kho A, Xu H, Wilkins C, Anton-Culver H, Boerwinkle E, Cicek M, Clark CR, Cohn E, Ohno-Machado L, Schully SD, Ahmedani BK, Argos M, Cronin RM, O’Donnell C, Fouad M, Goldstein DB, Greenland P, Hebbring SJ, Karlson EW, Khatri P, Korf B, Smoller JW, Sodeke S, Wilbanks J, Hentges J, Mockrin S, Lunt C, Devaney SA, Gebo K, Denny JC, Carroll RJ, Glazer D, Harris PA, Hripcsak G, Philippakis A, Roden DM; All of Us Research Program. (2022). The All of Us Research Program: Data quality, utility, and diversity. Patterns (N Y); 3(8), 100570. doi: 10.1016/j.patter.2022.100570. PMID: 36033590.

Cronin, R.M., Halvorson, A.E., Springer, C., Feng, X., Sulieman, L., Loperena-Cortes, R., Mayo, K., Carroll, R.J., Chen, Q., Ahmedani, B.K., Karnes, J., Korf, B., O’Donnell, C.J., Qian, J., Ramirez, A.H., All of Us Research Program Investigators.  (2021). Comparison of Family Health History in Surveys versus Electronic Health Records in the All of Us Research Program. Journal of the American Medical Informatics Association, 28(4):695-703. doi: 10.1093/jamia/ocaa315. PMID: 33404595. 

MHRN III Pilot Project 1: Stakeholder Views on Implementation of Suicide Risk Prediction Models

Grant Details

Funder: NIMH

Grant Number: U19MH121738

Grant Period: 09/24/2019 – 6/30/2021

Narrative: Age-adjusted suicide rates have been increasing in the U.S. over the past two decades. In 2017, more than 47,000 Americans died of suicide. Health care visits represent opportunities for suicide prevention because most individuals make an outpatient health care visit within a year of their suicide death and almost half have a visit within a month of their death. However, suicide risk is not always easily recognizable to clinicians—traditional clinical prediction is hardly better than chance. Predictive modeling that identifies patterns in “big data” from administrative and electronic health records has proven superior to clinical suicide risk prediction and routinely used suicide screening instruments. While predictive modeling holds promise for suicide prevention, how models should be implemented in routine clinical practice and the contextual factors that influence their use are understudied. The potential benefits of any risk prediction model, including those designed to identify suicide risks, are dependent on making sure that the models are deployed in a manner that does not harm patients, supports clinical care management, and is sustainable for health care delivery systems. We propose a pre-implementation pilot study in three settings, using one-on-one, in-depth interviews to explore health system administrators’, clinicians’, and patients’ expectations, experiences with, concerns, and suggestions for the early use of suicide risk prediction models. In the first setting, health system administrators are still considering what might be the best implementation approach. Interviews will help us understand how various stakeholder expectations match what is actually occurring in the two other settings where small pilot studies will be in process. One of these settings is planning outreach to high-risk patients independent of health care visits while the other is planning delivery of risk scores at the point of care. By studying different implementation strategies, we can compare relative advantages and disadvantages. We are particularly interested in effects on clinical workflows, clinician-patient relationships, and patient experiences. While there is an emerging literature supporting the promise of predictive models in health care, implementation factors and patient impacts have been largely ignored. Yet decisions regarding design and modeling methods and implementation processes should be driven by stakeholder requirements. Results of this pilot study will have important clinical implications and will not only inform large-scale implementation of suicide risk prediction models in health systems across the country but will also inform development of future risk prediction models and associated care processes tailored to stakeholders needs more generally (not limited to suicide risk). The long-term goals of this pilot project are to inform ongoing health system-level efforts to reduce suicide prevalence and prevent suicides by optimizing the use of suicide risk prediction tools.

  • Lead Site:
    • Overall PI: KPNW (Bobbi Jo Yarborough)
  • Participating Sites/Subcontractors:
    • HPI (site project lead Rebecca Rossom)
    • KPWA (site project lead Julie Richards; site PI Greg Simon)
  • Funder Contacts
    • Science Officer: Susan Azrin
    • Program Official: Michael Freed
    • Grants Management Official: Julie Bergerud

Documents

Funding Announcement

Notice of Award

Personnel Contact List

Current Status

We have completed and analyzed interviews with 10 health care administrators, 30 clinicians in behavioral health departments, and 62 patients across three health systems.

Summary of Findings

Administrators and clinicians

  • Use of a suicide risk prediction model and two differing implementation approaches were acceptable.
  • Clinicians desired opportunities for input on implementation decision-making.
  • They wanted to know how this manner of risk identification enhanced existing suicide prevention efforts.
  • They wanted additional training on how the models determined risk and why some patients appeared at risk while others do not.
  • Clinicians were concerned about lack of suicide prevention resources for newly identified patients.
  • They wanted clear procedures for situations when they could not reach patients or when patients remained at-risk over a sustained period.
  • They would like consolidated suicide risk information in a dedicated module in the EHR to increase efficiency.

Patients

  • Patients were generally supportive of suicide risk prediction models derived from EHR data.
  • Concerns included: 1) apprehension about inducing anxiety and suicidal thoughts, or 2) triggering coercive treatment, particularly among those who reported prior negative experiences seeking mental health care.
  • Participants engaged in mental health care or case management expected to be asked about suicide risk and largely appreciated suicide risk conversations
  • Patients preferred conversations to come from clinicians comfortable discussing suicidality.

Publications

Yarborough BJH, Stumbo SP. Patient perspectives on acceptability of, and implementation preferences for, use of electronic health records and machine learning to identify suicide risk. Gen Hosp Psychiatry. 2021 May-Jun;70:31-37. doi: 10.1016/j.genhosppsych.2021.02.008.

Yarborough BJH, Stumbo SP, Schneider JL, Richards JE, Hooker SA, Rossom RC . Patient expectations of and experiences with a suicide risk identification algorithm in clinical practice. BMC Psychiatry. 2022 Jul 23;22(1):494. doi: 10.1186/s12888-022-04129-1 .

Autism Registry

Project Name:
A Diverse Autism Registry for Effectiveness Studies     
Principal Investigator:
Lisa Croen, PhD
Principal Investigator Contact Information:
lisa.a.croen@kp.org
Funder
NIMH
Funding Period:
08/2011 – 07/2013
Abstract:
The overarching goal of this research proposal is to create a large, comprehensive and dynamic autism spectrum disorder (ASD) registry across several integrated health systems participating in the Mental Health Research Network (MHRN). This registry will enable rapid identification and enrollment of patients into future large-scale comparative effectiveness studies testing treatment, preventive and services interventions, as well as future pharmacogenomic and etiologic investigations. We will leverage the many unique resources provided by the MHRN, including: 1) the large and ethnically diverse population of children with ASD, who are representative of the communities served by the health plans, 2) comprehensive electronic medical records (EMR) systems which capture all patient interactions with the health plans, 3) the existing biospecimen repositories at some participating sites that will facilitate collection of genetic material from children and their family members, and 4) researchers with an established track record of engaging and enrolling patients in autism clinical research studies and extensive experience in conducting research using electronic databases.
Grant Number:
 U19MH092201
Participating Sites:
Kaiser Permanente Northern California, Oakland, CA (Lead Site)
Harvard Pilgrim Health Care, Boston, MA
Kaiser Permanente Georgia, Atlanta, GA
Kaiser Permanente Northwest, Portland, OR
Kaiser Permanente Southern California, Pasadena, CA
Investigators:
Lisa Croen, PhD (Lead)
Jeanne Madden, PhD
Ashli Owen-Smith, PhD
Frances Lynch, PhD
Karen Coleman, PhD
Virginia Quinn, PhD
Major Goals: Build a comprehensive and dynamic registry of children with an autism spectrum disorder receiving healthcare from one of the participating health plans. Collect survey data from a subset of these families regarding ASD services and treatments utilized and associated costs and perception of efficacy; quality of life; caregiver strain; coordination of care; and resource and educational needs. Establish a biospecimen bank for future research studies
Description of study sample:
The initial sample for development of the ASD registry included all members aged 0-17 with any recorded ASD diagnosis.  1272 chart reviews were completed to validate ASD diagnosis at 5 study sites, and 1155 families completed online surveys at 4 sites.
Current Status: Completed an ASD diagnostic validation study, ascertained children with ASD and built the registryCompleted the survey and collection of biospecimensCurrently, we are working on several manuscripts using registry and survey data. Work on this subject continues on a related MHRN project, “Maximizing Biospecimen Collection for Childhood Disorders
Study Registration: 
N/A
Publications:
Coleman KJ, Lutsky MA, Yau V, Qian Y, Pomichowski ME, Crawford PM, et al. Validation of autism spectrum disorder diagnoses in large healthcare systems with electronic medical recordsJ Autism Dev Disord.2015 Jul; 45(7):1989–96. doi: 10.1007/s10803-015-2358-0.Cummings, JR, Lynch, FL, Rust, KC, Coleman, KJ, Madden, JM, Owen-Smith, AA, et al. Health Services Utilization Among Children With and Without Autism Spectrum DisordersJ Autism Dev Disord. 2015 Nov; 46: 910-920. Doi: 10.1007/s10803-015-2634-z.Madden, J.M., Lakoma, M.D., Lynch, F.L. et al. Psychotropic Medication Use among Insured Children with Autism Spectrum DisorderJ Autism Dev Disord. 2016: 1-11. doi:10.1007/s10803-016-2946-7.Owen-Smith, AA, Bent, S, Lynch, FL, Coleman, KJ, Yau, YM, Pearson, KA, Massolo, ML, et al. Prevalence and predictors of complementary and alternative medicine use in a large insured sample of children with Autism Spectrum Disorders. Res Autism Spectr Disord. 2015 Sept 1; 17: 40-51. doi: 10.1016/j.rasd.2015.05.002.Becerra, T, Massolo, M, Yau, V, Owen-Smith, A, Lynch, F, et al. A Survey of Parents with Children on the Autism Spectrum: Experience with Services and TreatmentsThe Permanente Journal.  2017;21. doi: 10.7812/TPP/16-009.Becerra-Culqui TA, Lynch FL, Owen-Smith AA, Spitzer J, Croen LA. Parental First Concerns and Timing of Autism Spectrum Disorder Diagnosis. J Autism Dev Disord. 2018 Oct;48(10):3367-3376. doi: 10.1007/s10803-018-3598-6. PubMed PMID:29754290.
Resources:
N/A
Lessons Learned: Health plan databases can be used to identify a large number of children with a valid ASD diagnosis. In order to achieve a high response rate to the survey, more resource-intense and tailored contact strategies are necessary. Among families participating in the survey, a very high percentage agreed to donate a biospecimen
What’s next?
Working on new manuscripts.

Maximizing Biospecimen Collection for Childhood Disorders

Project Name:
Maximizing Biospecimen Collection for Childhood Disorders
Principal Investigator:
Lisa Croen, PhD
Principal Investigator Contact Information:
lisa.a.croen@kp.org
Principal Investigator institution:
Kaiser Permanente Northern California, Oakland, CA
Funder
NIMH
Funding Period:
07/2015– 06/2018
Abstract:
Autism Spectrum Disorder (ASD) and Bipolar Disorder (BD) are the two most common severe mental disorders affecting children and adolescents.  Few evidence-based treatments exist for either condition.  Furthermore, response to treatment varies widely between individuals – and no evidence exists to match individual children with specific treatments.  Research to address these gaps – both to develop new treatments and to more effectively match individuals with treatments – will isolating on identifying biomarkers or endophenotypes that can accurately identify more homogeneous subgroups of patients and genetically related family members.  This next generation of research will require collecting biospecimens from large and representative samples of families affected by childhood mental disorders and linking those specimens to comprehensive and longitudinal health records. The main aim of this pilot is to identify the best ways to maximize biospecimen collection from children with mental health conditions.  We plan to meet this goal by leveraging and extending our work during the first cycle of MHRN funding on the feasibility of biospecimen collection from families of children with Autism Spectrum Disorder (ASD).  We will use a mixed methods approach to develop low-cost, tailored recruitment and consenting processes that specifically address concerns about participating in studies using biospecimens from children with mental health conditions and that are appropriate to large and diverse populations.  
Grant Number:
U19MH092201
Participating Sites:                                       Kaiser Permanente Northern California, Oakland, CA (Lead Site)Kaiser Permanente Georgia, Atlanta, GAKaiser Permanente Northwest, Portland, OR
Investigators:
Lisa Croen, PhD
Ashli Owen-Smith, PhD
Frances Lynch, PhD
Major Goals: Specific Aim 1: Using data from a feasibility study of biospecimen collection conducted during the first cycle of MHRN funding and health system electronic medical record (EMR) data, we will examine the demographic and clinical characteristics of children and their families predicting successful recruitment of families to participate in biospecimen collection. Specific Aim 2: Explore knowledge, attitudes and beliefs about biobanking and perceived facilitators and barriers to biobank participation among parents of children with mental health conditions from diverse backgrounds using ethnographic/qualitative methods (interviews, focus groups). Specific Aim 3: a) Develop tailored methods for recruiting and consenting children/adolescents with mental health conditions and their families into a biospecimen bank for future research) Develop a strategy for pilot testing those methods across different pediatric mental health conditions.  Focus on low-cost methods appropriate for large and diverse populations
Description of study sample:
Fifteen families who participated in the MHRN1 study at each of the three study sites, for a total of 45 ASD interviews and 15 families who have a child with a bipolar disorder diagnosis at KPNC, for a grand total of 60 families.
Current Status:
Aim 1: Completed data analysis.Aim 2: Data analysis was completed. Manuscript has been drafted and is expected to be submitted for publication in the Fall of 2019.
Study Registration:15 ASD families have been recruited at KPGA15 ASD families have been recruited at KPNW30 families (15 ASD and 15 Bipolar) families have been recruited at KPNC.
Publications:
Manuscript for Aim 2 to be submitted for publication in the Fall of 2019.
Resources:
N/A
Lessons Learned:
In spite of the profound caregiver burden experienced by many parents of children with serious mental health-related issues, participants in the present study were eager to participate in research, in general, and in biospecimen research, in particular, as long as the research process involved trust, clarity, and flexibility. Researchers working with these populations need to make it easy for families to participate – even if that is less convenient and more costly for them – and should prioritize the sharing of knowledge gained by the research with participants at the end of the study.
What’s next? Working on new manuscripts.