Poorly planned mental healthcare transitions can lead to avoidable exacerbations in youth mental health problems, unnecessary cost expenditures and increased use of social services.1–8 Youth and their caregivers often experience the transition from child and adolescent mental health services (CAMHS) to community or adult mental health services (AMHS) negatively,9 10 specifically having a lack of information about post-CAMHS transition mental healthcare, being left out of the transition discussions, and not feeling prepared for the transition.11 It is not surprising then that up to 60% of youth get disconnected from mental healthcare during this transition, leading to avoidable negative mental health and social outcomes, wasted healthcare resources and unnecessary cost expenditures.3 5 12 These experiences have led policy-makers, administrators and other key stakeholders, including youth and their caregivers, to demand interventions to support more effective transitions.13 14
Evidence shows that adults who transition out of psychiatric care benefit from discharge planning interventions,15 particularly multicomponent interventions that are codesigned with patients and focus on patient discharge needs, follow-up and clear communication; all of which have been shown to reduce readmission16 and long-term impairment.17 Few mental health transition support interventions, however, exist for children and youth.18–20 A systematic review of structured paediatric transition interventions reported lack of focus on mental health settings.21 Promising transition models exist however, such as the use of transition teams and/or specialised navigators.22 23
Navigators are frequently recommended as a best practice in the mental health literature, given their expertise on transition practices, particularly working directly with youth to identify level of transition readiness and ongoing mental healthcare needs.24–26 Several quality improvement agencies,27–29 health professional bodies,30 31 government health sectors26 32 and youth advocacy groups13 have documented recommendations for CAMHS to community/AMHS transitions, which include an emphasis on the need for a key worker who takes on a coordinating role in the transitions process.
The Shared Management Framework has emphasised the use of navigator models in youth mental healthcare.23 Youth have reported the need for continuity of a therapeutic relationship with a trusted person (service provider) throughout the transition process as key to their transition success.9 33 Hospital-based CAMHS have used navigators to facilitate the complex transition out of hospital based acute care CAMHS to community CAMHS or AMHS and ensure continuity of care for transition aged youth.34
We summarised the core features of the navigator model in our recent review.34 Based on the clinical needs and goals of the youth and their caregivers, they are transitioned to a number of child or adult community programmes, for example, day treatment programmes, specialised supported school settings, family doctors and community-based counselling. Typically, navigators require a blend of qualifications (commonly a registered social worker), knowledge and clinical skills. They usually provide short-term (1–6 months) intensive support to bridge the youth using a combination of case management and system navigation.34 The navigator works with the youth to prioritise transition goals, assess and develop transition readiness, and collaborating with youth, their caregivers and clinicians to identify the level of need postdischarge and options for the receiving (transfer) agency. Data are captured at the programme level, with the navigator tracking: intervention components provided, rates of rehospitalisation, youth and caregiver’s satisfaction, length of time between discharge from hospital CAMHS and uptake in community or AMHS.
Despite the widespread implementation of the navigator model, there have been no evaluations of the role’s effectiveness,34 which makes scaling up problematic. Moreover, a recent review revealed a general lack of robust evaluations of child and adolescent mental health transition interventions.35 Our recent review34 highlighted an urgent need for a formal evaluation of the model considering clinical, programme and economic outcomes.
This study follows the Triple Aim Framework (Triple Aim) developed by The Institute of Healthcare Improvement.36–38 Triple Aim provides an overview of the importance of selecting outcome measures that represent three central aspects of interventions: population health outcomes, experience of care outcomes and utilisation and cost outcomes. The Canadian Association of Paediatric Health Centres proposed the Triple Aim as an emerging evaluation framework in their recent transition guidelines.39 As well, two systematic reviews of paediatric transition interventions used the Triple Aim to summarise outcomes reported in studies and recommend practical, feasible and common outcome measures for future transition in care evaluations.21 38 These recommendations were considered when selecting measures for this study.
The Navigator Evaluation Advancing Transitions (NEAT) study aims to collaborate with patients, caregivers, clinicians and navigators to evaluate the navigator model using the Triple Aim Framework. The aim is to address gaps identified in the literature by conducting a formal, patient-oriented, mixed-methods evaluation of the navigator model. As such, the study objective is to use patient-oriented research40 strategies in collaboration with youth, caregivers and clinicians to evaluate the navigator model.
Specifically, the NEAT study aims to answer the following research questions:
Using the Triple Aim framework, we will evaluate the navigator programme using a pre and post mixed-method study design. A mixed-method study design was selected as it is recommended for the evaluation of complex interventions.41 42 We will assess 45 youth and their caregivers (15 youth and 15 caregiver participants at each site) at baseline (referral to navigator) (T1) and 6 months (T2). Six months was selected as our review34 reported that youth were on average in the navigator programme for 1–6 months. At the 6-month follow-up (T2), all youth and caregiver participants who participated at baseline (T1) will be invited to participate in semi-structured qualitative interviews to better understand their experience with the navigator intervention. Clinicians will also be invited to participate in the qualitative interviews during this time. Our goal is to conduct a total of 24 interviews with youth, their caregivers, and clinicians (approximately eight participants from each group at each site).
Participants will be recruited from the child and adolescent psychiatry programmes at the Hospital for Sick Children (SickKids), North York General Hospital (NYGH) and Humber River Hospital (HRH) in Toronto, Canada. SickKids is a large tertiary hospital and NYGH and HRH are community hospitals. All three hospitals provide a variety of inpatient and outpatient mental health services, serve youth aged 16–18 years, and currently employ at least one navigator to facilitate transitions as youth are discharged from inpatient and outpatient care.
For both quantitative and qualitative components, the inclusion criteria for research participants are: Youth (1) between the age range 16–18 years, (2) receiving navigation services from the navigator programme, (3) able to speak, read and write English and (4) able to provide informed consent. Caregivers: (1) who are the caregiver of the youth receiving navigator services, (2) able to speak, read and write English and (3) provide consent to participate in the study. Clinicians: any clinician who has had a patient access the navigator programme. Navigators: who are responsible for implementing the navigator intervention at recruiting hospital sites.
We will simultaneously recruit youth, their caregivers, clinicians and navigators from three hospital sites over a 12-month period. All eligible youth who have a scheduled appointment with the navigator during the recruitment period will be invited to participate. During the first appointment, the navigator will inform the youth about a potential study opportunity. If the youth is agreeable, a research assistant (RA) will meet with the youth face-to-face and screen them based on the eligibility criteria. If found eligible, the RA will then provide further study information, answer any questions, and complete the informed consent procedure. Once written consent is obtained, the RA will set up a time to administer the baseline survey package. Baseline (T1) survey will be conducted after seeking consent and before the start of the navigator intervention.
Given that the study is recruiting youth with identified mental health problems, a clinical back-up will always be available to the RA and the youth will also be made aware of local service options should they need them.43 Based on our other ongoing research with similar populations, we will seek consent with youth, rather than assent, as youth aged 16–18 years have the capacity to consent to research even if a consenting caregiver is not available.44 Caregivers will be approached concurrently using the same process. Youth are permitted to participate in the study even if their caregiver is unable to or is not available.
Potential clinician participants will be identified by the navigator who will then inform the RA. Clinicians will be invited to participate in the study via email and, if willing, the RA will meet the clinicians in person to complete the informed consent process.
Navigators from all three hospital sites will be invited to participate in the NEAT study. The RA will directly contact the navigators and, if agreeable, the RA will explain about the study and complete the consent process. All participants will maintain their right to withdraw from the study at any time.
Youth and caregiver participants will complete self-report study measures at baseline and at 6 months. At each time point, time required for data collection is expected to be 30 min. Youth and caregiver participants can complete the study measures either onsite (in person) using paper assessment packages, or online using a secured and encrypted web application called Research Electronic Data Capture (REDCap; http://project-redcap.org/)45 where study participants can log on and complete study measures. An RA will be available to answer any questions, either in person or via email, phone, or text (if using REDCap). The web forms will be developed, tested, operated, and maintained by the study team. Clinician participants will complete semistructured interviews either onsite (in person) or over the phone, while navigator participants will complete a checklist online at the 6-month follow-up (T2).
In accordance with the Triple Aim framework, three outcomes will be studied—(1) population health outcomes, (2) experience of care outcomes and (3) utilisation and cost outcomes. The primary outcome is transition readiness measured by Transition Readiness Assessment Questionnaire (TRAQ46;) at baseline and 6 months. The secondary outcomes include mental health trajectories, transition experience and service utilisation assessed at baseline and 6 months.
Transition readiness is selected as the primary outcome measure as it has been recommended in a recent Cochrane Review47 and has been used as the primary outcome in several child and youth health transition interventions.48 49 Further, in keeping with the principles of participatory action research,50 51 youth, caregivers and clinicians were involved in the selection of these measures and endorse their relevance to this study. Table 1 summarises the outcome domains and corresponding data collection tools and timelines for each participant group.
| Domain | Source of data collection | T1 | T2 | ||||
| Youth | Caregiver | Youth | Caregiver | Clinician | Navigator | ||
| Sample description | Demographic questionnaire | ✓ | ✓ | ✓ | ✓ | ✓ | |
| Health outcomes | Transition Readiness Assessment Questionnaire | ✓ | ✓ | ✓ | ✓ | ||
| Columbia Impairment Scale | ✓ | ✓ | ✓ | ✓ | |||
| Youth Self-Report | ✓ | ✓ | |||||
| Child Behaviour Checklist | ✓ | ✓ | |||||
| Adolescent Alcohol and Drug Involvement Scale | ✓ | ||||||
| Experience of care | Core Components of Effective Youth Transitions | ✓ | ✓ | ✓ | |||
| Satisfaction with Mental Health Navigator Tool | ✓ | ✓ | |||||
| Qualitative Interviews | ✓ | ✓ | ✓ | ||||
| Utilisation and cost | Assessment of Quality of Life-6D | ✓ | ✓ | ||||
| Health and Social Services Utilisation | ✓ | ✓ | ✓ | ✓ | |||
Demographic data will be collected from youth, caregivers and clinicians to describe key characteristics of the sample. The demographic form used in the Longitudinal Youth in Transition Study study52 will be used, as it was cocreated with a youth and caregiver advisory team. For youth and caregiver participants, demographic data will be collected at baseline and 6 months.
For youth participants, we will collect their age (in years), sex at birth, current gender identity, total length of service (years/months) in CAMHS, primary self-reported mental health diagnosis, cultural group, current living situation, if anyone attends appointments with them, and current school and/or work hours. For caregiver participants, we will collect their relationship to the youth participant, age of their youth who is receiving mental health and addiction services (in years), and current school and/or work hours. Lastly, for clinician participants, we will collect their current gender identity, occupational role/clinical background, years of clinical experience, relationship to the youth participant (ie, primary clinician) and length of time treating the youth participant.
Four measures will be used to assess health outcomes. Youth and caregivers will report their readiness (related to skills, knowledge and self-efficacy) to transition out of services using the 20-item TRAQ.46 The instrument is scored on a 5-point Likert scale; total scores (/100) will be generated, with higher scores indicating greater readiness and ability to negotiate transitions in care. The TRAQ was found to be the best transition-readiness tool53 54 with the strongest reliability (Cronbach’s α.82–.9355) for use with youth and caregivers and recommended as a standardised assessment of transition programmes.56
To assess functioning, youth and caregivers will complete the Columbia Impairment Scale (CIS),57 a 13-item scale which provides a global measure of impairment in four major areas of functioning: interpersonal relations with family and friends, broad psychopathological domains, functioning in job or schoolwork, and use of leisure time. Both the youth (Cronbach’s α.78) and caregivers (Cronbach’s α.89) version of the CIS has been shown to have good internal consistency58 and test–retest reliability.57
To measure mental health symptoms, youth will complete the Youth Self-Report (YSR), a 119-item scale, which measures psychiatric symptoms and adaptive functioning.59 Caregivers will complete the Child Behaviour Checklist/ 6–18 (CBCL/6–18), a 113-item scale that assesses behavioural and emotional problems in children and adolescents.59 Both the YSR and CBCL/6–18 yield scores on an eight-syndrome scale.59 Youth will be assessed for their alcohol and substance use using an adapted version of the Adolescent Alcohol and Drug Involvement Scale (AADIS). This 13-item checklist assesses youth’s alcohol and substance use pattern on a continuum ranging from no use to severe dependence. The AADIS is a revised tool based on Adolescent Alcohol Involvement Scale (AAIS) and the adolescent drug involvement scale ADIS.60 Internal consistency is satisfactory for ADIS for youth (Cronbach’s α=0.85)60 and that of AAIS was (Cronbach’s α=0.84).61
This outcome will be measured using two measures. Youth and caregivers will complete the Satisfaction with Mental Health Navigator Tool, a 11-item scale, adapted from the Navigation Satisfaction Tool.62 This tool measures youth and caregiver satisfaction with the navigator model. Youth, caregivers, and navigators will complete a 27-item Core Components of Effective Youth Transitions (CCEYT) checklist at the 6-month follow-up (T2), with each item scored as ‘yes’ (the youth was satisfied that component occurred), ‘no’ (they were not satisfied, or the component did not occur), and ‘Unsure’. This checklist aims to capture core components of stages of effective transitions identified in the literature63 and is validated via a National Delphi consensus study.64 Given the core features of the navigator model aligns with the 27 CCEYT components, this checklist will permit the evaluation of the degree to which each youth perceived they received the navigator intervention components. The navigators will be asked to complete the CCEYT checklist to understand their perspectives on the extent to which the core components of the navigator intervention are being implemented for each youth participant.
In order to capture health service utilisation, youth and caregivers will report on the Health and Social Service Utilisation (HSSU) measure,65 66 a structured interview that assesses use of health programmes and services in the past 6 months with emphasis on both physical and mental health services (ie, psychiatrist, general practitioner, private therapist, community drop-in agencies, online/telephone counselling, school counsellor, etc), lab services (ie, blood work) and current (prescribed and non-prescribed) medications. This instrument of ambulatory care utilisation has been empirically validated. To assess, QoL, youth will complete the Assessment of Quality of Life-6D (AQoL-6D), a 20-item assessment of six domains of QoL: independent living, relationships, mental health, coping, pain and senses. The six domains can be combined to form a single global QoL factor67 68 with strong internal consistency (Cronbach’s α=0.81) across hospital and community samples.69 The QoL-6D will be used to calculate quality adjusted life years (QALYs). In addition, we will collect comprehensive information on the costs of setting up and delivering this information. Taken together, this information (ie, interventions costs, health services utilisation and costs, and QoL) will be used to explore the possibility of undertaking a cost utility analysis of the navigator model in accordance with the Canadian Agency for Drugs and Technologies in Health guidelines.70
Semistructured, in-depth interviews will be conducted with youth, caregivers and clinicians to obtain narrative descriptions of the navigator intervention. For all three participant groups, the interviews will focus on the participants’ experiences and satisfaction with the navigator intervention (see online supplemental file 1). Example questions for youth include: ‘What has been your experience receiving care from the navigator?’ and ‘What aspects of the navigator care/intervention did you find most, and least, helpful?’ Qualitative interviews will be conducted at 6 months and will take place in person, or via videoconferencing as needed (COVID-19 has limited on-site research capabilities in some hospital settings). Interviews will be conducted by a research team member with at least a master’s education who has both clinical expertise and experience in conducting and facilitating qualitative interviews with youth and caregivers in mental health settings. It is expected each interview will take 30–45 min and will be audio recorded and transcribed verbatim by an RA.
10.1136/bmjopen-2021-051190.supp1bmjopen-2021-051190supp001.pdf
All analyses will be undertaken using 9.4 SAS.71 Effect size and 95% CI will be reported for each outcome. A two-sided p<0.05 will be defined as statistically significant.
Descriptive statistics will be used to summarise the characteristics of study sample at baseline. Continuous variables (eg, age) will be summarised using a mean and SD, while ordinal and categorical variables (eg, gender identity) will be expressed using frequency counts and percentages.
Primary research question: Change in TRAQ scores from baseline to 6 months will be analysed using a paired sample t-test. Based on previous research, it is assumed TRAQ scores will be normally distributed; however, if skewed, non-parametric testing will be used (ie, Wilcoxon-signed rank test). Mean difference will be calculated along with 95% CI. Cohen’s D will be calculated as a measure of effect size. Secondary research question: A similar analysis strategy as above will be used to assess change over time for the CIS and YSR/CBCL. AADIS checklist will be assessed using descriptive statistics only. The three HSSU composite measures will be analysed using: (1) time in weeks between discharge and first community visits using t-test (if normally distributed); (2) total number of unscheduled emergent visits and (3) total number of rehospitalisations using descriptive statistics. The AQoL-6D will be analysed both for a simple total score (by adding the response of each item) and QALYs will be analysed following the developers analytical recommendations, in short, by multiplying life years by an index of utility measured on a 0–1 scale.72 Last, measures of intervention satisfaction, will be completed at 6 months. Total scores (continuous) for the CCYET Core Components checklist will be analysed for the entire sample using descriptive statistics.
We will use Braun and Clarke’s73 thematic analysis methodology to analyse qualitative data. NVivo software will be used for data management,74 and more than one researcher will analyse the qualitative data in order to enhance the credibility of the findings.75 The first author (KC) and a research team member will independently read and reread interview transcripts to become familiar with the content and to look for meaning and patterns. They will establish initial codes by identifying concepts and ideas present in the text. They will then independently examine the data at a broader level, by reviewing initial codes and grouping them into themes. Themes will be reviewed to ensure that they meaningfully represent connected codes and are distinguishable from one another. Finally, themes will be defined and refined within the context of the data set as a whole in preparation for presentation of analysis. After completing this process independently, the two researchers will come together and share themes to ensure agreement and consistency. Levels of agreement of themes will be tracked, and where disagreements exist, the researchers will review the codes and corresponding transcript text and come to a consensus. In line with the patient-oriented research framework,40 76 youth and caregivers will be presented the initial coding framework and corresponding transcript text for feedback and interpretation. Adjustments to the coding framework will be made in collaboration with the youth and caregivers.
The use of the parallel mixed-methods design in this study means that the quantitative and qualitative data are collected and analysed separately, with each providing a unique lens and understanding of the experiences and satisfaction with the navigator model. The results of the two sets of analysis will then be linked, combined and integrated into meta-inferences, with the goal being a more comprehensive understanding of the intervention using analytical strategies describe by Teddlie and Tashakkori.77
The sample size calculation was based on the primary outcome—change in TRAQ scores from T1 to T2 assessed using a paired t-test. We anticipate a mean TRAQ score of 3.54 (SD 0.71) (out of a possible 5) at baseline, as reported by a similar population in our other ongoing studies (ages 16–18 years)78 and an anticipated mean score of 4.04 post-intervention resulting in a mean difference of 0.5 (as reported in a recent Cochrane review of transition interventions among youth47). Using G*Power,79 with α=0.05, it was determined that a sample size of 34 youth would provide 80% power to detect such a difference. Assuming a 20% attrition at the 6-month follow-up, we will enrol at least 41 youth in three sites, that is, 15 at each of the three sites.
The sample size will be approximately 15–20 for each participant group. We aim to have representation of at least eight youth, eight caregivers and eight clinicians from each of the three participating sites. KC and a research staff will conduct preliminary thematic analysis after approximately 15 interviews from each group to determine whether thematic saturation has been reached and add additional interviews, as necessary. This is in line with recommended guidelines for qualitative interview sample size determination during data collection.80 81
All study data will be managed in accordance with the Tri-Agency principles of digital data management (Government of Canada) and according to Personal Health Information Protection Act (PHIPA) guidelines.82 All study data from each recruitment site will be deidentified, coded numericaly and password protected in the case of electronic data and entered into (REDCap; http://project-redcap.org/).45
The study started enrolling participants in March 2021. The estimated completion date for study enrolment is March 2022.
With specific focus on participatory action research,50 51 the NEAT study is engaging youth, caregivers and a core team of researchers and knowledge users (KUs) in every stage of the evaluation as part of the Expert Advisory Committee (EAC). To date, this has included engagement on the recruitment strategies and methodologies, the selection of outcome measures and developing interview guides. Additionally, one youth with lived experience and two caregivers are collaborators on the funding application. Youth and caregivers will also be involved in interpretation of findings and knowledge dissemination activities. Further, using integrated knowledge translation methods, we would ensure that the study goals and outputs are relevant to the needs of patients and KUs, feasible and reach a wide audience.83
This study protocol has been approved at the lead research sites: the University of Toronto (UofT; REB#: 39046), and the Hospital for Sick Children (SickKids; REB#: 1000066139). REB approval has been received by North Youth General Hospital (NYGH; REB#: 20–0023)] and is currently being sought at the third recruitment site (Humber River Hospital). The EAC and coinvestigators will continue to meet regularly throughout the project to develop and then implement an end-of-project knowledge mobilisation plan.84 Provisional components are as follows: (1) A webinar will be developed in collaboration with the EAC to identify organisations, teams and regions that may be used for future implementation and research; (2) Peer-reviewed open-access publications and (3) workshops and/or presentations will be held at key KU meetings (eg, academic and community conferences, mental health agencies). Members of the EAC, coinvestigators and site collaborators will be invited to copresent and coauthor the project findings. The results will be published using the Criteria for Reporting the Development and Evaluation of Complex Interventions (CReDECI)-2 criteria for reporting of interventions in healthcare.85
The authors would like to thank the Expert Advisory Committee members and collaborators (Christina Bartha, Crystal Edwards, Tania Fiacco, Barbara Hansmann, Jennifer Holmes-Harronitis, Alexia Jaouich, Krista Lemke, Kimberly Moran, Michele Sparling, Vicky Stergiopoulos). We would like to thank the Youth Engagement Facilitator and Youth Advisor Members of the Youth Engagement Initiative team at CAMH who provided feedback on the study design, study measures and process. We also extend our thanks to the CAMHS services, clinicians and community mental health agencies who are collaborating in this study. Lastly, the authors would like to acknowledge the study research assistant, Soha Salman, for her assistance with the operationalisation of the study.
Not required.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
Tweets
Mixed-methods study protocol for an evaluation of the mental health transition navigator model in child and adolescent mental health services: the Navigator Evaluation Advancing Transitions (NEAT) study
Twitter
Facebook
WhatsApp
LinkedIn
