Setting and design

The study was conducted during the yearly 2-day continuing medical education conference ‘Boerhaave Progress and Practice’39 for primary healthcare physicians (Leiden, The Netherlands, 14 December 2018). Each year, a range of somatic and mental health topics are covered by specialists within 25 min time slots. During one of those slots, we presented the survey supported by PowerPoint slides that additionally carried embedded subtitled audio fragments of the vignettes (60–64 s each). A total of 275 GPs were registered. Attendees were seated with a device that digitally recorded their anonymous responses. As such, data collection was not subject to Dutch law governing medical research involving human subjects, nor to European law on general data protection and privacy.

Procedure and measurement

Attendees were introduced with (1) that they would hear five case descriptions of children with psychosocial problems in the form of a report resulting from a few consultations, and would see three recurring questions that reflect their daily reality concerning what the condition might be and what could be done about it. (2) They were asked to base their considerations on the available information only. We aimed to use gender-neutral names for the vignettes to prevent that gender confounds decision-making. In an independent coding of 25 gender-neutral names by six coders, at most three names were labelled by half of the coders as gender-neutral. Given this low agreement and taking into account discrepancies in literature on gender differences and recognition,29 36 we mentioned (3) that the named cases were intentionally gender-neutral. Attendees were also informed (4) that by responding they voluntarily and anonymously agreed to analysis of their responses for scientific purposes. Preceding the first vignette, the recurring questions were verbally introduced and shown (table 1). Thereafter, the questions were shown sequentially, each with a progress bar displaying the response time set to a maximum of 10s.

Table 1

Survey items as shown and verbally introduced

Questions and options as shown	Questions and options verbally introduced as
A1 What is the main complaint?	Where do you think this description fits in? To which symptom-profile would you ascribe this vignette?
1. Typical development	1. Option one if it is probably an example of typical development.
2. Behavioural problems	2. Option two for difficult behaviour: examples include aggressive behaviour or antisocial behaviour.
3. Complaints regarding establishing contact	3. If problems likely indicate an autism spectrum disorder, you can choose option three.
4. Mood problems	4. Mood problems and problems that could be related to depressive disorders.
5. Somatic complaints	5. For physical symptoms choose option five, also if a problem might be psychosomatic in nature.
6. Eating problems	6. Option six for eating problems and probable eating disorders.
7. Anxiety-related complaints	7. Option seven for problems related to anxiety and anxiety disorders.
8. Complaints regarding attention and activity	8. Option eight for attention-related complaints that might indicate attention deficit hyperactivity disorder or attention deficit disorder.
9. Complaints related to the experience of a traumatic event	9. Option nine for problems related to the experience of a traumatic event.
A2 Second complaint group?	If you would like to add a second problem to the main complaint groups you can select it here.
10. No second complaint group	10. Please choose option 10 on your voting machine if you do not see another complaint.
A3 Where should this child ideally go?	Then, for each vignette, the triage question: where can this child and the family get the most adequate professional support?
1. Practice nurse   2. Local youth teams   3. Generalised mental healthcare   4. Specialised mental healthcare   5. Somatic healthcare/hospital
Where do you think that children with this type of complaint are best helped? B1. Behavioural problems B2. Complaints regarding establishing contact B3. Mood problems B4. Somatic complaints B5. Eating problems B6. Anxiety-related complaints B7. Complaints regarding attention and activity B8. Complaints related to the experience of a traumatic event	The eight mental health groups will be shown again. Please indicate for each of these how you generally tend to refer children when you suspect these complaints.
1. Watchful waiting   2. Practice nurse   3. The local youth teams   4. General mental healthcare   5. Specialised mental healthcare	If you often tend to first monitor such complaints for a while, you can opt for ‘1’. If you often ask your mental health practice nurse to become involved option ‘2’, and so on, up to the point where you feel that a more intense treatment option is adequate. Then, you can choose option ‘5’: specialised mental healthcare.

Survey items A1–A3 were presented after each vignette. Given their relevance for early recognition, the response options for the identification questions (A1 and A2) were formulated in terms of ‘complaints’ and ‘problems’ rather than ‘disorders’. We presented two identification questions to provide sufficient opportunity to identify an AD in the mixed vignettes. The third recurring question queried how the respondents would have referred such a child or family in daily practice, and was presented with response options that reflect the Dutch ‘stepped care’ services. The practice nurse is a mental health professional (MHP) who works in general practice. The option ‘hospital’ was presented since the vignettes also depicted physical problems. As we introduced the study as one about children with psychosocial problems, the option watchful waiting was not presented for the vignettes. The survey items B1–B8 were presented after the last survey item A3. Here, the option watchful waiting was presented following our interest in whether ADs are viewed as something children grow out of.

Vignette development

In order to quantify to what extend GPs consider anxiety in their early diagnostic opinion, we constructed vignettes that capture various symptoms (online supplementary file A). Guided by knowledge of the clinical presentation of ADs and the prevalence of common mental health disorders, we had decided to build five vignettes around the combined presentation of anxiety with, respectively, physical complaints, difficult behaviour, mood, developmental and school attendance problems. On the level of the key aspect mixed-anxiety, each vignette represented symptoms related mostly to one of these domains while also depicting symptoms of anxiety that do not overlap with these mental health problems.

bmjopen-2019-035799supp001.pdf

To theoretically support the process, we first consulted clinical handbooks and questionnaires.40–42 Relevant symptom expressions and idioms were obtained from actual clinical referral letters written by GPs of children and adolescents diagnosed in specialised MHC. This linked text-data enabled working back and forth from children’s complaints mentioned by their GPs and their descriptive diagnoses, and hence facilitated a controlled development of vignettes that are also natural. The extracted descriptions were grouped into the five domains and an initial set of 11 vignettes (as advised)43 was formed. In an iterative process five vignettes were selected, further refined and for readability purposes named as V1-Somatic, V2-Behavioural, V3-Mood, V4-Developmental, V5-School Attendance. In each vignette, more cues were given for anxiety compared with any other specific disorder group (eg, in V2-Behavioural vignette, we counted the total number of cues as 13, with eight cues that could be related to anxiety, five to behavioural problems, three to attention-hyperactivity and trauma, two to somatic and one to autism). To mimic actual consultations, the vignettes also included contextual factors43 such as academic or home functioning or background information about the GP–child relationship. Case age and vignette length had a limited range (10–12 years, 165–172 words) to avoid possible confounding effects.43

During the final stage of vignette development, GPs affiliated with Leiden University Medical Centre were invited to participate in an individual pilot interview (six interviews held in November 2018). GPs were asked to think aloud and provide verbal feedback while reading the transcript of each vignette and answering the identification and referral questions. We also asked GPs about the realism of the vignettes: they responded positively and felt that the descriptions corresponded to their practice. The pilot led to minor changes, most concerning the ‘contextual aspects’,43 for example, initially we alternated mothers and fathers as the parent who went along to the consultation. Several GPs evaluated this as odd, hence we changed it to ‘mother’.

Validation of vignettes

To validate the recognisability of anxiety, 24 MHPs were invited to participate in a pilot presentation, of whom 11, all with extensive clinical experience (median 15–19 years), agreed to participate. All MHPs were employed at Curium-LUMC, a university-affiliated inpatient and outpatient psychiatric clinic for children and adolescents. We surveyed MHPs using the same procedure as in the GP-survey, with the exception of written responses and two additional questions regarding their psychiatric specialisation. The presentation resulted in 55 answers concerning the first identification question, and 53 answers on the second identification question. In line with our aim to develop mixed-anxiety vignettes, MHPs selected different disorder groups: anxiety (n=22), mood (n=19), attention-hyperactivity (n=14), behavioural (n=12) and somatic (n=10) problems were selected frequently, and typical development was not selected (online supplementary file A). Anxiety was the most or second most frequently selected disorder group in V1-Somatic, V2-Behavioural and V4-Developmental, and the third most frequently selected in V3-Mood and V5-School Attendance.

Outcome definition and analysis

Recognition rate was defined as the ratio of the total number of times anxiety was selected and the total number of times anxiety could have been selected. GPs’ and MHPs’ recognition was compared following Newcombe’s method 1044 using the Epi package45 in the R-statistics environment. This method tests for independence as in the χ² test, yet provides a robust method to estimate CIs. GPs’ selection rate of the different disorder categories and the referral indications they made for the vignettes were analysed using descriptive statistics. Whether the likelihood of an MHC referral was lower when anxiety was recognised in a vignette was tested in a logistic multilevel regression using the lme4 package.46 That GPs would report an MHC referral for ADs less often compared with referral for behavioural, developmental and mood problems was also tested in a logistic multilevel regression, with disorder groups (anxiety vs behaviour, development and mood problems) as the repeated measures. Respondents with missing responses on more than half of the survey were excluded from all analyses. Data were visualised using the ggplot2 package.47

Patient and public involvement

Patients and/or the public were not involved in the design, or conduct, or reporting or dissemination plans of this research.

Participants

A total of 299 attendees answered at least one survey question. Of these, 239 reported being a GP. Data from 10 GPs were excluded due to missing responses. As a missing response on both identification questions of a vignette occurred 17 times, the number of times anxiety could have been selected was 1128 (229*5–17).

The distribution of the sample’s gender and years of experience was similar to the national population48 (table 2).

These demographics showed no association with the number of times anxiety was recognised, the referral indications GPs made for each vignette, or with their reported referral preferences for ADs (online supplementary file B).

Table 2

Demographics of the study sample and the national population

	Study sample n=229	Population n=9798
Gender
Male	116 (50.7%)	4799 (49.0%)
Female	95 (41.5%)	4999 (51.0%)
Unknown	18 (7.9%)	–
Experience in years	Age distribution in national population
0–2	4 (1.7%)	<30	76 (0.8%)
3–5	22 (9.6%)	30–34	976 (9.9%)
6–9	27 (11.8%)	35–39	1428 (14.6%)
10–14	16 (7.0%)	40–44	1518 (15.5%
15–19	35 (15.3%)	45–49	1396 (14.2%)
>20	124 (54.1%)	>50	4357 (44.5%)
Unknown	1 (0.4%)	Unknown	56 (0.6%)

Recognition

By selecting 167 times anxiety in the 1128 possibilities to opt for a specific disorder group, GPs reached a recognition rate of 14.8%. GPs were less likely to recognise anxiety compared with MHPs (OR=0.26, 95% CI 0.15 to 0.46, figure 1). Whereas 55.9% (n=128) of the GPs labelled one or more vignette as anxiety-related, over two-fifths did not in any. On average, GPs often selected behavioural, mood, developmental and trauma-related problems, and typical development (table 3). GPs and MHPs did not differ in their selection rate of these disorder groups (online supplementary file B).

Figure 1

The upper figure depicts MHPs’ and GPs’ recognition rate of anxiety. MHPs selected anxiety in nine of their 55 responses on the first identification question and in 13 of their 53 responses on the second identification question. GPs selected anxiety in 84 of their 1060 responses on the first identification question and in 83 of their 1067 responses on the second identification question. Differences between the two groups were statistically significant overall (OR=0.26, 95% CI 0.15 to 0.46), in the first identification question (OR=0.44, 95% CI 0.21 to 0.93), and in the second identification question (OR=0.26, 95% CI 0.13 to 0.50). The lower figure depicts the percentage of GPs and MHPs that recognised anxiety in none, one, two, three, four or all five vignettes. The difference between GPs’ and MHPs’ recognition frequency was statistically significant, χ²_{(5, 240)}=42.94, p<0.001. GPs, general practitioners; MHPs, mental health professionals.

Table 3

GPs’ selection rate of each disorder category

	V1-Somatic		V2-Behavioural		V3-Mood		V4-Developmental		V5-School Attendance
	First	Second	First	Second	First	Second	First	Second	First	Second
Anxiety	4.0	1.9	22.4	14.2	4.6	9.8	3.4	6.1	5.8	6.9
Trauma	0.9	1.4	46.3	29.2	0.5	–	1.0	0.5	9.7	24.1
Mood	1.3	1.9	1.0	2.8	52.8	22.8	–	2.8	38.6	11.6
Somatic	8.8	14.6	–	0.5	0.9	0.9	0.5	–	9.2	5.1
Eating	5.3	10.4	–	–	–	–	–	–	4.3	7.9
Autism	10.6	11.8	1.0	–	32.6	27.9	8.3	17.9	4.8	4.2
Attention-hyperactivity	7.1	3.3	6.3	11.8	4.1	13.0	67.6	16.5	1.9	1.9
Difficult behaviour	29.6	14.2	22.0	32.1	3.7	6.0	15.7	22.2	6.8	3.7
Typical development	32.3	12.7	1.0	–	0.9	0.9	3.4	7.5	18.8	4.2
No second complaint group		29.2		9.4		18.6		26.4		30.6

Selection rate of each disorder group shown in percentages, per identification question (first and second) and per vignette (V1–V5). Missing responses resulted in the following sample sizes in V1-Somatic=226 and 212, V2-Behavioural=205 and 212, V3-Mood=218 and 215, V4-Developmental=204 and 212, V5-School Attendance=207 and 216, with the first value depicting the sample size in the first identification question, and the latter the second identification question of each vignette.

GPs, general practitioners.

Exploration of recognition per vignette revealed a major difference between GPs and MHPs for V1-Somatic (GPs: 5.7%, MHPs: 54.5%). Differences were not statistically significant for V2-Behavioural (GPs: 33.8%, MHPs: 45.5%), and for V3-Mood (GPs: 13.7%, MHPs: 18.2%). GPs recognised anxiety significantly less often than MHPs in V4-Developmental (GPs: 9.0%, MHPs: 36.4%) and in V5-School Attendance (GPs: 11.9%, MHPs: 45.5%).

Referral

When asked how they would refer a child with a similar profile to the vignettes, the majority responded that they would keep the child in general practice (practice nurse=40%, figure 2). GPs who indicated that they would refer most often chose primary healthcare (local youth teams=24%, primary MHC=13%). The vignette which most often triggered referral to MHC was V3-Mood.

Figure 2

GPs’ referral decisions following each of the five vignettes. Sample size was as follows in V1-Somatic=213, V2-Behavioral=220, V3-Mood=224, V4-Developmental=215, V5-School Attendance=211. GPs, general practitioners; MHPs, mental health professionals.

Those GPs who recognised anxiety reported referral of a child to MHC slightly less often compared with those who did not recognise anxiety (table 1, online supplementary file B). However, multilevel analysis revealed that the likelihood of a referral to MHC was not significantly related to the recognition of anxiety (OR=0.70, 95% CI 0.42 to 1.18) (table 4).

Table 4

Referral to mental healthcare by those GPs who recognised anxiety and those who did not

		Other referral options	Mental healthcare
V1-Somatic	Selected	13 (100%)	0 (0%)
	Not selected	192 (96.0%)	8 (4.0%)
V2-Behavioural	Selected	59 (78.7%)	16 (21.3%)
	Not selected	105 (73.9%)	37 (26.1%)
V3-Mood	Selected	24 (80.0%)	6 (20.0%)
	Not selected	137 (71.0%)	56 (29.0%)
V4-Developmental	Selected	17 (89.5%)	2 (10.5%)
	Not selected	166 (86.9%)	25 (13.1%)
V5-School Attendance	Selected	22 (91.7%)	2 (8.3%)
	Not selected	170 (91.9%)	15 (8.1%)
Mean V1–V5	Selected	88.0%	12.0%
Not selected	83.9%	16.1%

Selection frequency of each referral option per vignette partitioned by GPs who selected anxiety and who did not. Data revealed no significant associations between recognition of anxiety and referral to MHC (OR=0.70, 95% CI 0.42 to 1.18, p=0.19). An MHC referral includes referral to primary MHC and specialised MHC, other referral options include the mental health practice nurse in general practice, local youth teams and somatic healthcare (selection rates of each specific referral option are depicted in online supplementary file B).

GPs, general practitioners; MHC, mental healthcare.

When asked about their general referral preferences, a majority reported a tendency to refer children with suspected ADs to primary MHC (figure 3). More GPs reported treatment of suspected anxiety in MHC (OR=1.79, 95% CI 1.35 to 2.37) compared with suspected developmental, behavioural or mood problems (OR=0.53, 95% CI 0.39 to 0.72). Analysis per disorder group revealed that GPs’ reported referral preferences for anxiety, did not differ from their preferences for mood problems. A referral to MHC was reported less often for difficult behaviour, problems related to autism, attention-hyperactivity and (psycho-)somatic problems and more often for eating problems and trauma (online supplementary file B).

Figure 3

GPs’ reported referral preferences for each of the eight disorder groups. Excluding cases with missing responses left the following sample sizes: anxiety n=224, trauma n=217, mood n=220, somatic n=212, eating problems n=219, autism n=213, attention-hyperactivity n=214, difficult behaviour n=216. GPs, general practitioners; MHC, mental health care.

Strengths and limitations of the study

To the best of our knowledge, this is the first study to investigate GPs’ interpretations of the various problems typical in children with ADs. We surveyed a large sample but we do not have insight into the representativeness of the attending GPs. Nevertheless, the years of experience and gender distribution of the sample were similar to the national GP population, these demographics did not show an association with the outcome measures.32 Also, literature is inconclusive in whether GPs’ attitudes influence decision-making.30 31 The presentation of the vignettes to all participants at once yielded a risk of carryover effects. It also incited a game aspect that may have decreased observer effects and improved access to participants’ honest and direct considerations.43 A limitation is the small number of MHPs and the multiple statistical comparisons made between GPs and MHPs. Notwithstanding the explorative nature of the study on the level of the individual vignettes, we applied conservative Bonferroni corrections to avoid false positives. MHPs selected anxiety infrequently in the mixed anxiety-depression vignette. This low recognition rate might be a consequence of similar number of cues for anxiety (8) and depression (7) in this vignette, the large conceptual overlap between anxiety and depression and that MHPs might also be more attuned towards depression.49 GPs could not indicate watchful waiting for the vignettes in this study overestimating the likelihood that GPs take any action when presented with children similar to the vignettes. In conversations with GPs. we had found that in case of any ambiguity they would make the rational decision for further enquiry. Given our aim to gain insight in their tendencies, we decided to prevent selection of this safe option. Nevertheless, it is unlikely that addition of this option would change the results considering referral to MHC. Those who would have selected watchful waiting now probably selected the practice nurse. A related caveat is the interpretation of the findings as representing recognition and referral in everyday practice. Decision-making in general practice is shaped by a complex interplay of multiple individual and systemic factors over time. With this novel approach, we aimed to isolate the extent to which GPs mind anxiety in their early diagnostic opinion. In this regard, a major strength of the study was the use of actual patient medical records to create the vignettes. This enabled wording similar to that of GPs, improving the construct and external validity of the case descriptions and diminishing uncontrollable variability.50

Comparison with other studies

Overall recognition of anxiety was within the lower range of what is found in literature (9%–60%).27 28 51 This might be a consequence of the mild severity portrayed in the vignettes, a factor limiting recognition of all disorder groups.32 This design, however, does not explain why GPs selected the other mental health problems yet failed to recognise anxiety proportionately. In V1-Somatic depicting a consultation for recurring earaches, for example, GPs picked up on the probability of behavioural, somatic or autism spectrum-related problems. They were also sensitive to whether typical development was depicted. Nonetheless, the portrayal should have also raised the prospect of an AD, a possibility that was recognised by very few GPs. GPs recognised anxiety sufficiently in one vignette. This being the vignette that literally mentioned the core feature of anxiety (‘fears’) supports that unfamiliarity with symptom presentation might be the driving factor behind the low recognition rates.

The recognition rate was realised by just over half of GPs. Others did not select anxiety in any vignette. A question is what underlies these individual differences. Previous studies also found no association of recognition with gender or experience of the professional, and literature is not one-sided about the role of attitude.31 Observations we made during the pilot with six GPs suggest that differences might be caused by how they ‘perceive, read and interpret’37 presenting symptoms. For instance, all GPs commented about the word ‘temperamental’, yet many neglected ‘timid’ although both words were in the same sentence. GPs also varied in the problem areas they gave attention to. Most GPs expressed a focus on school and home burden (eg, ‘School is a benchmark for me and that is going well. If it wasn’t alarm bells would be going off’). By contrast, many did not reveal a need to enquire further with children themselves or about leisure activities and social relations. If a GP tends to piece out some symptom-profiles and neglect others based on context,52 53 this might be of particular consequence for the recognition of ADs characterised by a covert and less disruptive profile.

An important finding concerned the final vignette which portrayed a child with social AD. In response to this vignette, GPs frequently selected typical development and the mildest referral options. Although a similar response pattern was obtained for the first vignette, both the vignette developers and MHPs evaluated the final vignette as a more severe case description. This finding is in line with studies suggesting low recognition of social anxiety.29 35 54 Recognition of social anxiety by professionals is of particular importance since these patients are known to under-report symptoms, and since diagnosis-specific interventions are advised in clinical guidelines.55

Contrary to hypothesis, GPs did not prove less likely to prefer intervention in MHC for ADs compared with problems related to difficult behaviour, developmental disorders or depression. Results of this normative part of the study contrasts with clinical studies demonstrating that a substantial proportion of children with ADs are unlikely to be referred to MHC.25 30 56 This disparity with clinical practice might be related to factors other than GPs’ conscious evaluations, such as strain on parents and their treatment desires that differ between disorders.57 Another explanation might be a discrepancy in GPs conscious evaluations on the one hand, and their skills and implicit decision-making tendencies on the other hand. Our data support this explanation to some extent, although to our knowledge, no previous study examined the evaluations and implicit tendencies GPs hold towards treatment of children with ADs. We found a discrepancy in GPs’ responses in that the majority of the GPs reported to consider a referral when they suspect ADs, but generally chose management in primary healthcare rather than referral to MHC in the vignettes. This comparison of reported referral preferences to the referral indications made for the vignettes might be invalid if GPs assumed a higher severity when asked about referral preferences versus the mild severity depicted in the vignettes. However, in both, we emphasised that the survey considers probable mental health problems, including subthreshold levels of severity. The variations in GPs’ answers between disorder groups supports that they understood the question as it was meant. Also, GPs were inclined to refer the vignette scenario with symptoms that overlapped with mood problems to MHC, in agreement with their reported normal practice. This leaves open whether their threshold to evaluate anxiety as an AD differs from when they consider mood problems as depression,54 yet supports our hypothesis that differences in the likelihood of a referral may depend on symptom presentation. Notably, during the individual pilot interviews, we observed a focus on symptoms related to depression ‘because you would not want to leave it untreated’ and varying heuristics GPs used when evaluating the relevance of symptoms.

Meaning of the study

Moving beyond sensitivity as the endpoint of diagnostic accuracy, the results suggest that the repeatedly found underrecognition of anxiety goes back to the initial considerations GPs make. Our findings confirm the conjecture that recognition of ADs is often confounded by their overlap with other common and well-known mental health problems and similarity with typical developmental variations.26 However, this finding per se does not distinguish between the alternate possibilities that GPs prioritise other disorder groups or lack sufficient knowledge of the presentation and prevalence of child anxiety. In fact, it has been suggested that there might be a prioritisation issue given the seemingly low burden and impact of ADs.38 In the current study, the presentation of the vignettes using subtitled audio fragments precluded the possibility of rereading and elaborate evaluation. By verbally presenting vignettes that do not indicate a distinctive mental health disorder, we aimed to circumvent knowledge of diagnostic criteria and the question of a primary or differential diagnosis. The study results therefore imply that, even before rational prioritisation, there could be a significant bottleneck in first interpretation that might have otherwise led to early recognition. Anxiety might not resonate sufficiently as the likely problem when GPs encounter children with ADs. Leaving aside the discussion of whether it is a GP’s task to assess the specifics of a child’s mental health problem, we argue that it is worthwhile to improve knowledge of early symptom presentation in children with problems that are relatively poorly recognised by parents and teachers. GPs face the difficult task to recognise mental health problems in a limited amount of consult time, yet report difficulties with the use of structured assessment methods in clinical practice.58 To avail the benefits of their unguided decision-making, it is necessary that they become more attuned and aware of the prevalence, the breadth and relevance of typical signs of anxiety. Understanding how professionals comprehend and act on presenting symptoms is important, even more so in light of the high base-rate of ADs,8 their early age of onset (11 years)18 and risk of persistence into adulthood.1–3 The finding that majority of the GPs reported to prefer an MHC referral indicates that they do acknowledge anxiety as a treatment demanding problem, and leaves opportunity for targeted improvements.

Unanswered questions and future research

Why did some GPs recognise anxiety in multiple vignettes whereas others did not? Another open question concerns how GPs would have responded had the referral option ‘no action needed’ been available for the vignettes. Causal inferences as regards symptom presentation and subsequent recognition and referral could be made if the many symptoms associated with ADs could be systematically varied using a factorial vignette design.43 If combined with qualitative methods, future studies could further enhance our understanding of the decision-making process.