Child and Adolescent Psychiatry and Mental Health
BioMed Central
Integrating clinical and research training in child psychiatry: fifteen-year outcomes of a federally supported program
Volume: 14
DOI 10.1186/s13034-020-00328-4
•
•
•
• Altmetric

### Notes

Abstract

BackgroundThe Albert J. Solnit Integrated Training Program (AJSP) is an educational initiative designed to prepare physician-scientists for independent careers in the investigation and treatment of childhood psychiatric disorders.MethodsWe compared fifteen cohorts (each representing a consecutive year of matriculation) of AJSP trainees and graduates (n = 30) to peers who were comparably ranked in our original match lists but ultimately pursued residency programs elsewhere (n = 60). Outcomes of interest between the two groups included professional affiliation, as measured by: (1) membership in the American Academy of Child and Adolescent Psychiatry (AACAP); and (2) certification by the American Board of Psychiatry and Neurology (ABPN), as well as three domains of research productivity: (1) Competitive awards received from AACAP; (2) Publication-related metrics derived from the National Library of Medicine (NLM); and (3) Federal grant funding from the National Institutes of Health (NIH).ResultsAJSP participants were more commonly affiliated with AACAP and board certified in CAP. AJSP graduates and trainees outperformed their control group peers in several research outcomes: (1) Receipt of AACAP awards and number of awards per recipient were higher, and time to first award shorter in the AJSP than in the control group; (2) AJSP participants had more publications in PubMed, more first-authored publications, a higher h-index, and a shorter time to first publication than participants in the control group; and (3) NIH K- or R-series funding success rate was higher among AJSP participants (p < 0.05 for all comparisons).ConclusionsA program designed to support the development of clinician-scientists specifically dedicated to childhood mental health needs has been successful in fostering scientific creativity, productivity and independence. The expansion and replication of similar training initiatives will be an in important step forward to address the high level of morbidity and mortality associated with child and adolescent psychiatric disorders.

Keywords
Calhoun, Bloch, Stubbe, Leckman, and Martin: Integrating clinical and research training in child psychiatry: fifteen-year outcomes of a federally supported program

We must shift our emphasis away from shepherding our trainees from one requirement or rotation safely onto the next. If we are to deliver on the latent promise of our discipline, we must train a new generation capable of redefining it. Our educational mission should be less about guiding and monitoring our pilgrims’ progress than about aggressively promoting our pipeline’s promise [ 1].

## Background

Concerns about the declining numbers of physician-scientists have existed for several decades. The ‘clinical investigator as an endangered species’ was first mentioned in 1979 in the eponymous paper by future NIH director James Wyngaarden [2]. Twenty years later, Leon Rosenberg aptly called physician-scientists as both ‘endangered and essential’ and continued to identify some of the underlying challenges and possible solutions [3]. In 2016 the National Institutes of Health (NIH) convened a series of workshops to update programmatic, system-wide solutions to the enduring challenge of clinician-investigator recruitment and retention [4]. The percentage of US physicians engaged in patient-oriented research has steadily declined for the past 30 years, from a peak 4.7% in the 1980s to 1.5% as of 2012 [5, 6]. A variety of reasons have been posited for this decline, including: an increasing portion of students with a large academic debt, an increase in the amount of time required to prepare for a research career; and the perception by physicians that they may not be competitive with PhDs.

For psychiatry, the decreasing number of physician-scientists is especially problematic. Viewed in light of challenges such as the public health costs of mental illness, addiction and an aging population, as well as opportunities to utilize scientific advances to improve prevention, early intervention and treatment of psychiatric disorders, the need for psychiatrist-researchers is particularly urgent. In addition to the common concerns about personal economic disincentives and long duration of training, medical students and psychiatry residents face limitations in the availability of appropriate research education and training opportunities.

The need in the area of child mental health is particularly great given the shortage of child and adolescent psychiatrists (CAPs), the prevalence and diversity of child and adolescent mental disorders, their public health costs, and their potential long-term impact on society. One quarter of the U.S. population is under the age of 18, and at least 13% of these children and adolescents (some 15 million individuals) have diagnosable psychiatric disorders [79]. Nine to 13% of U.S. children and adolescents meet the definition of a ‘serious emotional disturbance’ [10]. However, in 2017, only about one in nine emotionally disturbed U.S. children and adolescents received any mental health services [11]. It is estimated that there is only one CAP per 1807 children and adolescents who are in need of mental healthcare [12]. As of 2016, no state in the US has what professional groups would deem a sufficient number of CAPs to serve children in need [12].

#### Peer reviewed publications

We collected bibliographic information from the US National Library of Medicine using PubMed (pubmed.gov), and citation metrics from the Data Citation Index of the Web of Science (Clarivate Analytics, webofknowlledge.com). For each participant we determined: (a) Number of publications, starting 1 year after program matriculation; (b) Number of first-authored publications; (c) h-index, defined as the largest number of articles cited at least h times [27]; (d) Cumulative citations; and (e) Mean and highest impact factor (IF) of the journals cited. We then calculated time from program matriculation to first published article.

#### Federal grant funding

We obtained information on federal grant funding using the National Institute of Health’s Research Portfolio Online Reporting Tools (RePORT, report.nih.gov). For each participant we determined: (a) Receipt of K-series award; (b) Receipt of R-series award; and (c) Total dollar amount over funded period. Finally, we calculated time from program matriculation to first conferred award.

### Statistical analysis

We compared variables of interest across AJSP and control groups using Chi square tests for nominal variables, and independent sample t-tests for continuous measures. We used non-parametric tests for two positively skewed variables that were not normally distributed: (a) Rank order in the match list; and (b) Number of publications. In both instances we used the independent samples median test with Yates correction. For time-to-outcome analyses we used the non-parametric Kaplan–Meier estimator. We plotted its results as the 1—cumulative survival function over time and compared the mean times to event using the log rank (Mantel-Cox) test for equality of survival distributions. We conducted all analyses using SPSS 25.0 (Armonk, NY).

## Results

Thirty participants matriculated into the AJSP in the 15 years between 2004 and 2018. They were comparable in rank order to sixty participants who ultimately matched into other accredited programs in the US. Median and interquartile range [IQR] rank order was not different between the AJSP (3 [4]) and control groups (5 [3]; t = 2.1, df = 1, ns).

Demographic and other descriptive characteristics of the AJSP group (including a sixteenth cohort that matriculated in 2019) are summarized in Table 1. Sex distribution has been balanced among the 32 participants, eight of whom (25%) are members of under-represented minorities (URM; four African American, three Latinx, one Native American). One-third of participants have an advanced degree, separate from their MD (16 PhD, 4 MPH or equivalent). Of those degrees, 15 had been conferred prior to matriculation, and 5 were earned as part of academic work during the course of the AJSP. One half of eligible participants received financial support in the form loan relief through the NIH Pediatric Research Loan Repayment Program. Because we had no comparable information on these variables for the control group, we did not make intergroup comparisons.

Table 1
Albert J. Solnit Integrated Training Program (AJSP) Participants
n%
Unique participants (enrolled 2004–2019)32100
Currently Enrolled (2020)1238
Demographic characteristics
Women1753
African American413
Latinx39
Native American13
Upon matriculation1547
NIH Pediatric Research Loan Repayment Program (among 24 eligible)1250

Table 2 summarizes our a priori defined outcomes between the AJSP and control groups for the first 15 cohorts. With respect to professional affiliation, AACAP membership was higher in the AJSP than in the control group (97 vs 32%, p < 0.001). General psychiatry board certification among eligible participants was the same between groups (83%), but CAP certification was higher among AJSP participants (85 vs 53%, p < 0.05). Times to board certification did not differ between groups (4.8 ± 1.3 years for general psychiatry, and 6.6 ± 3.6 years for CAP).

Table 2
Professional affiliation and research productivity outcomes between AJSP and control groups
AJSP
n = 30
Control
n = 60
Statistic
Professional affiliation
AACAP membership: n (%)29971932x2df=1 33.95***
ABPN certification
Psychiatry: n/eligible (%)20/248340/4883x2df=1 0.0
CAP: n/eligible (%)17/208521/4053x2df=1 6.06*
AACAP research awards
Awards
Any: n (%)22731220x2df=1 24.20***
Travel to annual meeting: n (%)15501017x2df=1 11.08**
Pilot research: n (%)175747x2df=1 27.95***
Junior investigator: n/eligible (%)6/20300/400FET***
Number: mean (SD)2.10.81.40.7t df=1 2.08*
Peer-reviewed publications
Any: n (%)27904575x2df=1 2.81
Number: mean (SD)14.127.46.716.5t df=1 4.07*a
Median (IQR)5.516.02.07.0
First authored: mean (SD)4.96.42.45.1t df=88 2.03*
h index: mean (SD)9.010.05.05.8t df=86 2.39*
Cumulative citations: mean (SD)5271303192470t df=86 1.76
Impact factor: mean (SD)6.97.85.55.1t df=69 0.33
Highest: mean (SD)15.016.013.617.0t df=69 0.98
NIH grant funding
K series: n/eligible (%)7/20355/4013x2df=1 4.22*
R series: n/eligible (%)3/20152/405x2df=1 1.75
R series (CAP-related): n/eligible (%)3/20150/400FET*
Dollar amount (×100,000): mean (SD)9.36.79.17.4t df=10 0.05

AJSP Albert J. Solnit Integrated Training Program, IQR interquartile range, FET Fisher’s Exact Test, AACAP American Academy of Child and Adolescent Psychiatry, ABPN American Board of Psychiatry and Neurology

aIndependent samples median test with Yates correction

* p < 0.05 **p < 0.01 ***p < 0.001

AJSP graduates and trainees outperformed their control group peers in most research outcomes: First, receipt of AACAP research awards was higher in the AJSP than in the control group (73 vs 20%, p < 0.001). All three types of research awards were more common in the AJSP group (p ≤ 0.001 for each). The number of awards per recipient was higher among the AJSP participants (2.1 ± 0.8 vs 1.4 ± 0.7, p = 0.04). Next, AJSP participants had more publications in PubMed than control group participants (14.1 ± 27.4 vs 6.7 ± 16.5, p = 0.044), more first-authored publications (4.9 ± 6.4 vs 2.4 ± 5.1, p = 0.046), and a higher h-index (9.0 ± 10.0 vs. 5.0 ± 5.8, p = 0.019). Cumulative citations and impact factor metrics did not differ across groups. Finally, NIH K- or R-series grant funding success rate among graduates was higher among AJSP than control group participants (35 vs 13%, p = 0.04). Neither the number of years awarded, nor the dollar amounts funded differed statistically between the groups. Of note, of the five R-series grants awarded, the three in the AJSP group were on topics related to CAP or developmental science; the two in the control group were not (p = 0.04).

Figure 1 depicts visually the time-to-first of three outcomes of interest between the AJSP and control groups. Years to first AACAP award was shorter in the AJSP than in the control group (4.7 ± 0.4 vs 6.9 ± 0.9; log rank = 35.05, df = 1, p < 0.001). Years to first peer-reviewed publication was likewise shorter (2.9 ± 0.6 vs 5.3 ± 0.6; log rank = 5.44, df = 1, p = 0.02), as was time to first K-series award (8.6 ± 0.5 vs 9.6 ± 0.2; log rank = 4.66, df = 1, p = 0.03).

Fig. 1
Years to first of three outcomes of interest between AJSP and control groups.
: AJSP (Albert J. Solnit Integrated Training Program),
:Control group. x axes denote time (in years); y axes denote fraction of subjects with outcome of interest, calculated as 1—the survival function obtained though the Kaplan–Meier estimator. p values calculated with log rank (Mantel-Cox) tests for equality of survival distributions. *p < 0.05, ***p < 0.001

## Discussion

AJSP graduates and trainees outperformed their control group peers in most outcome measures of interest:

We incorporated membership as a proxy variable to determine trainees’ commitment to careers in child and adolescent psychiatry, particularly after graduation. There is no other streamlined way to determine career trajectory. The three-fold higher AACAP membership rate in the AJSP group indicates its participants are and remain committed to working in the field. This is no trivial point, given that many promising would-be investigators in child psychiatry end up doing research in other fields given the long training duration and lag time necessary for subspecialized work with children and adolescents. Our findings suggest, albeit indirectly, that graduates of the AJSP are more likely to become active contributors not just to research in psychiatry, but specifically within the area of child and adolescent psychiatry. The high success rate in applying for AACAP awards, the larger number of awards received, and the shorter duration to first award all suggest that the AJSP has been effective in ‘socializing’ its participants from early on in their training into the work of, and opportunities provided by the Academy.

We tracked ABPN certification as a means of determining trainee’s clinical competency. AJSP participants were more likely to get board-certified in CAP compared to control peers, and despite the emphasis on research in the residency program, AJSP trainees did not take significantly longer to receive their initial board certifications compared to their control group peers. It should be noted that board certification may not be pursued by some graduates—particularly those opting for translational research careers, or those going to academic positions in which there may be less time pressure to obtain certification. Despite these caveats, AJSP graduates were still more likely to become board certified in CAP.

The majority of participants in both groups published peer-reviewed work indexed by the National Library of Medicine. However, participants in the AJSP were considerably more prolific, as reflected by a greater number of published and first-authored articles, and a shorter time to first publication. The fact that mean and highest impact factor did not differ between the groups indicates that participants in the AJSP were not simply publishing more articles, but publishing in comparable quality venues. The h-index of AJSP participants was higher, suggesting more heavily cited and visible scientific output. This finding is notable in that the h-index, like the impact factor, has shortcomings that include the time required to accrue citations [28]. Indeed, cumulative citations did not differ between the two groups, showing the long lag time needed for citation accrual.

A commonly used ultimate metric of scientific independence is receipt of NIH funding, particularly R-series grants. Participants in the AJSP had higher success rates in obtaining K-, but not R-series funding. The average dollar amount did not differ between groups. Given the low number of R-series awards (five) among this sample of 90 participants, and the long lag time to obtain a first R (an average of 10.6 ± 1.5 years from program enrollment) it behooves educators and funding agencies contemplating disbursement of funds to consider alternative metrics of scientific productivity in the shorter term.

### Active components of the AJSP

We would have been delighted to take into the AJSP any one of the applicants in our overall sample of ninety. Indeed, the sixty applicants who did not come to Yale were just as highly ranked (by us) as those who stayed. Each and every one of them was an exceptional individual with an impressive track record even that early in their medical education. Given this comparability in applicant qualifications, and given that many of the programs they went on to have several commonalities with our program (such as resources, grants, and infrastructure), we conclude that the differences in outcomes we found are attributable to the unique characteristics of the AJSP. We go on to outline what we consider to be these ‘active ingredients’.

In developing and refining the AJSP iteratively over its first 15 years in operation, we have followed a dozen guiding principles we consider integral to its success:

• Early identity formation. Trainees incorporate direct experience in caring for children and families as they develop as both clinicians and independent physician-scientists;
• Mentorship and career development. Trainees are assigned a research faculty mentor from their first year onwards. Mentors are accomplished investigators with a sustained record of competitive research funding and active research programs. Mentors have a major responsibility for supervising the trainee, providing assessment and constructive feedback, documenting the trainee’s research progress and performance, and assisting with career development and application for a K award. Participants work closely with their research mentor and residency training director to tailor an appropriate sequence of clinical training and research education and experience;
• Integrative program structure. The program integrates research with clinical training, and child and adolescent clinical training with adult training by structuring these experiences concurrently and using shared group learning and faculty supervision to foster integration. Unlike traditional training models in psychiatry, both research and child psychiatry training begin early and continue throughout the residency. AJSP trainees are also encouraged (but not required) to have clinical mentors that guide them to develop particular clinical skills of interest across the training period;
• Optimal focus on child psychiatry. Wherever possible, child psychiatry rotations are substituted for ones in adult psychiatry, if/as permitted by ACGME and ABPN requirements for both adult and child and adolescent psychiatry. For example, pediatric medicine is scheduled as the required primary care medicine rotation;
• Foundation of core clinical training. The AJSP provides a full range of inpatient and outpatient experiences that support the acquisition of fundamental clinical skills in adult and child psychiatry. Residents achieve competencies in all six areas identified by the ACGME: medical knowledge, patient care, practice-based learning, interpersonal and communication skills, systems-based practice, and professionalism. They are clinically evaluated using the Milestones of the ABPN. Their strong clinical foundation serves as the basis for evidence-based clinical practice and the development of advanced research skills toward a career as independent investigators;
• Evidence-based perspective. The principles and practice of Evidence-Based Medicine (EBM) anchor the curriculum and training experiences in both adult and child psychiatry. Regularly scheduled EBM seminars build skills in evidence-based clinical practice;
• Early research immersion. Intensive immersion in clinical psychiatry research starting during the second year fosters early professional identity development as an investigator and is expected to reduce attrition from long-term commitment to research careers;
• Formal research training. Optimally, training in the AJSP includes coursework leading to a Ph.D. or master’s degree, if such a degree was not already acquired and/or as determined by a trainee’s learning needs assessment. Concurrent formal research training that is separately supported is available to AJSP trainees through the Investigative Medicine Program or the Yale Department of Epidemiology and Public Health. This formal training can begin as early as the third year;
• Instruction in responsible conduct of research (RCR). A robust approach to RCR includes formal educational activities supplemented by lectures, workshops and substantial face-to-face discussions. In addition, trainees become familiar with policies and procedures addressing academic misconduct, conflict of interest and conflict of commitment, human subject research protection, and (when relevant), institutional animal care and use;
• Comprehensive research experience. The AJSP provides a research experience that is comprehensive in terms of time, formal curriculum, mentorship, structured evaluation and feedback. These components are essential for professional growth and development. Over the course of training, AJSP trainees are guided through progressive, supervised research experiences, from critical appraisal of the literature, literature reviews and secondary data analyses, through increasingly complex research projects, independent study design and grant-writing, culminating in the submission of an application for a career development (K-series) award in their final year;
• Commitment to the enhanced recruitment of under-represented minorities (URMs). The term URM refers to those ethnic or racial groups that are underrepresented in the field of medicine, including African American/African, Hispanic/Latinx, and Native American or Pacific Islander. Eight out of 32 (25%) of AJSP trainees identify as URMs. This fraction is considerably higher than 9%, the AAMC’s reported national average of URM physicians practicing medicine [29]. Multiple reports have also highlighted the severe need for more physician-scientists with URM backgrounds, with only 7% of NIH grant awardees being URMs [30]. In this context, recruitment of URMs with the intent of improving diversity in the physician and the CAP physician-scientist workforce has been an important programmatic goal; and
• Debt repayment. Scheduled research time of at least 80% in the final 2 years of the AJSP qualifies trainees for the NIH Loan Repayment Program. Thus far, one half of eligible residents enrolled in the AJSP have been able to secure debt loan repayment through this mechanism.

## Limitations

Our study has inherent limitations, beginning with its focus on a single program, which may limit generalizability, particularly to settings with a smaller child psychiatry presence or more limited research infrastructure. In addition, our sample was relatively small, and a 15-year window is not sufficient to assess longer-term outcomes, like those pertaining to NIH R-series funding. We were not able to collect information on self-reported ethnicity or on current academic or tenure-earning positions for all members of the control group, which limited our ability to determine how our participants and graduates fared comparatively. Finally, we were not able to get individual-level data regarding federal debt relief for control group participants, as the LRP dashboard (dashboard.lrp.nih.gov) provides only state-aggregated information.

## Conclusions

In summary, we found that a program specifically designed with the aim of providing specialized training for physician-scientists committed to careers in child and adolescent psychiatry has been able to meet its aims 15 years since inception. Critical to the program’s fiscal wellbeing are two federal grants that combined provide approximately one quarter of its overall funding (17% and 8% through the R25 and T32 mechanisms, respectively). At a time of uncertainty regarding NIH funding, we consider our findings informative in setting priorities and confirming a solid return on investment. Specifically, the 2.8 million dollars in R25 funding since 2004 has already yielded 4.6 million dollars in new grant funding. Aside from the fiscal bottom line, the AJSP has led to the formation of a unique group of clinician-scientists with remarkable scientific creativity, innovation and output. We look forward to continuing to refine the program and are committed to its longevity. We are hopeful that other programs may consider replicating or expanding the AJSP approach, something that the Universities of Colorado and Vermont have already started doing. There is a pressing public health need for these and other innovative approaches to enhance the training of clinician-scientists devoted to ease the suffering of children and adolescents with psychiatric illnesses.

## Notes

Publisher's Note:

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

## Acknowledgements

The authors dedicate this work to Drs. Albert J. Solnit (1919–2012) and Donald J. Cohen (1940–2001), the third and fourth directors of the Yale Child Study Center (1966–1983 and 1983–2001, respectively). In loving memory.

## Authors’ contributions

All authors contributed to the design of the study; AC collected and harmonized the data; AM conducted the statistical analyses; AC and AM wrote the first draft of the manuscript; all authors contributed to the interpretation of the findings and made substantial contributions toward the final manuscript. All authors read and approved the final manuscript.

## Funding

Supported by NIMH awards R25 MH077823 and T32 MH018268, and by the Riva Ariella Ritvo Endowment at the Yale School of Medicine.

Not applicable.

## Ethics approval and consent to participate

This study was approved by the Yale Human Research Protection Program’s Institutional Review Board (#2000026883).

## Consent for publication

As a secondary analysis of publicly available data, it was deemed exempt and did not require informed consent.

## Competing interests

The authors declare that they have no competing interests.

## References

1.

Martin A, Pipeline Promise (Foreword). Training (Christopher Varley, Guest Editor) Child and Adolescent Psychiatric Clinics of North America 2007;16(1): xiii–xv.

2.

Wyngaarden JB. The clinical investigator as an endangered species. N Engl J Med 1979. 301: 23, pp.1254-1259, doi: 10.1056/NEJM197912063012303

3.

4.

Hall AK, Mills S, Lund PK. Clinician-investigator training and the need to pilot new approaches to recruiting and retaining this workforce. Acad Med 2017. 92: 10, pp.1382-1389, doi: 10.1097/ACM.0000000000001859

5.

National Institutes of Health. Physician-scientist workforce working group report. 2014. https://acd.od.nih.gov/documents/reports/PSW_Report_ACD_06042014.pdf. p. 14.

6.

Garrison HH, Deschamps AM. NIH research funding and early career physician scientists: continuing challenges in the 21st century. FASEB J. 2014. 28: 3, pp.1049-1058, doi: 10.1096/fj.13-241687

7.

Costello EJ, Mustillo S, Erkanli A, . Prevalence and development of psychiatric disorders in childhood and adolescence. Arch Gen Psychiatry 2003. 60: 8, pp.837-844, doi: 10.1001/archpsyc.60.8.837

8.

National Institute of Mental Health: National survey tracks rate of common mental disorders among American youth. 2009. http://www.nih.gov/news/health/dec2009/nimh-14.nhm. Accessed 1 Jan 2020.

9.

Merikangas KR, He J-P, Burstein M, . Lifetime prevalence of mental disorders in US adolescents: results from the national comorbidity study-adolescent supplement (NCS-A). J Am Acad Adolesc Psychiatry. 2011. 49: 10, pp.980-989, doi: 10.1016/j.jaac.2010.05.017

10.

Olfson M, Druss BG, Marcus SC. Trends in mental health care among children and adolescents. New Engl J Med 2015. 372: , pp.21, doi: 10.1056/NEJMsa1413512

11.

Kaiser Family Foundation. Percent of children (ages 3-17) who received any counseling from a mental health professional. Retrieved from Health Care Expenditures per Capita by State of Residence, 2017. http://www.kff.org. Accessed 1 Jan 2020.

13.

National Research Council (US) and Institute of Medicine (US) Committee on the Prevention of Mental Disorders and Substance Abuse Among Children, Youth, and Young Adults: Research Advances and Promising Interventions; O’Connell ME, Boat T, Warner KE, editors. Washington: National Academies Press; 2009.

14.

US Department of Health and Human Services Health Resources and Services Administration & Maternal and Child Health Bureau. Mental health: A report of the Surgeon General. Rockville, MD: US Department of Health and Human Services, Substance Abuse and Mental Health Services Administration, Center for Mental Health Services, and National Institutes of Health, National Institute of Mental Health; 1999.

15.

Underwood LA, Washington A. Mental illness and juvenile defenders. Int J Environ Res Public Health. 2016. 13: 2, pp.228, doi: 10.3390/ijerph13020228

16.

National Alliance on Mental Illness. Juvenile Justice. 2020. https://www.nami.org/Learn-More/Mental-Health-Public-Policy/Juvenile-Justice.

17.

Bhat V, Leong KW, Lee J, Voineskos D, Daskalakis ZJ, Lam RW, Jollant F. Research-track programs for residents in psychiatry: a review of literature and a report of 3 Canadian experiences. Can J Psychiat 2014. 59: 5, pp.268-275, doi: 10.1177/070674371405900506

18.

Posporelis S, Sawa A, Smith GS, Stitzer ML, Lyketsos CG, Chisolm MS. Promoting careers in academic research to psychiatry residents. Acad Psychiatry 2014. 38: 2, pp.185-190, doi: 10.1007/s40596-014-0037-7

19.

Van Schalkwyk GI, Katz RB, Resignato J, Van Schalkwyk SC, Rohrbaugh RM. Effective research mentorship for residents: meeting the needs of early career physicians. Acad Psychiatry 2017. 41: 3, pp.326-332, doi: 10.1007/s40596-016-0625-9

20.

Hamoda HM, Bauer MS, Demaso DR, Sanders KM, Mezzacappa E. A competency-based model for research training during psychiatry residency. Harvard Rev Psychiatry. 2011. 19: 2, pp.78-85, doi: 10.3109/10673229.2011.565249

21.

Mezzacappa E, Hamoda HM, Demaso DR. Promoting scholarship during child and adolescent psychiatry residency. Acad Psychiatry 2012. 36: 6, pp.443-447, doi: 10.1176/appi.ap.11110199

22.

Dingle AD. Child psychiatry: what are we teaching medical students?. Acad Psychiatry 2010. 34: , pp.175-182, doi: 10.1176/appi.ap.34.3.175

23.

Kishore A, Sun KL, Guth SE, Kolevzon A, Martin A. Child and adolescent psychiatry perceptions and career preference: participation in a national medical student conference improves outcomes. J Am Acad Child Adolesc Psychiatry 2020. 59: 1, pp.3-7, doi: 10.1016/j.jaac.2019.07.949

24.

Himmelstein R, Guth S, Enenbach M, Gleason MM, Stevens H, Glowinski A, Kolevzon A, Martin A. Increased psychiatry match rates follow exposure to a mentorship program: a multisite retrospective cohort analysis. Acad Psychiatry. 2020. , doi: 10.1007/s40596-020-01210-3

25.

Stubbe D, Martin A, Bloch M, . Model curriculum for academic child and adolescent psychiatry training. Acad Psychiatry 2008. 32: 5, pp.366-376, doi: 10.1176/appi.ap.32.5.366

26.

Albert J. Solnit Integrated Training Program (AJSP) at the Yale Child Study Center. Program website available online at: https://medicine.yale.edu/childstudy/education/psychiatry/integrated/. Accessed 14 Dec 2019.

27.

Hirsch JE. An index to quantify an individual’s scientific research output. Proc Natl Acad Sci 2005. 102: , pp.16569-16572, doi: 10.1073/pnas.0507655102

28.

Hunt GE, Cleary M, Walter G. Psychiatry and the Hirsch h-index: the relationship between journal impact factors and accrued citations. Harvard Rev Psychiatry 2010. 18: , pp.207-219, doi: 10.3109/10673229.2010.493742

29.

Health eCareers. 2018. Minorities in medicine: a look at physician diversity. https://www.healthecareers.com/article/healthcare-news/minorities-in-medicine-a-look-at-physician-diversity.

30.

National Institutes of Health. Underrepresented racial and ethnic groups, 2019. https://extramural-diversity.nih.gov/diversity-matters/underrepresented-groups. Accessed 1 Jan 2020.

Citing articles via
https://www.researchpad.co/tools/openurl?pubtype=article&doi=10.1186/s13034-020-00328-4&title=Integrating clinical and research training in child psychiatry: fifteen-year outcomes of a federally supported program&author=Amanda Calhoun,Michael H. Bloch,Dorothy Stubbe,James F. Leckman,Andrés Martin,&keyword=Research training,Clinical training,Long-term outcomes,Child and adolescent psychiatry,Scientific independence,&subject=Research Article,