Cultural adaption and validation of the German version of the diagnostic thinking inventory (DTI-G) / Ein Instrument zur Erhebung diagnostischer Kompetenz: Validierung und kulturelle Adaptation des Diagnostic Thinking Inventory (DTI-G)

The adaption and translation of the DTI were performed according to Beaton et al. (2000), involving two forward translations in German language, synthesis of the translations, two back translations in English and consolidation of the German versions involving the original developer. Bilingual translators whose mother language is German produced the two forward translations. One of them is a professional translator (university diploma), and the other is a speech and language therapist. The pre-final version of the DTI-G was then pretested in the target group. The pretest focussed on relevance, comprehensibility and identification of unclear formulations or questions that are irrelevant for the target group. Twelve cognitive interviews were conducted using the think-aloud method (Willis, 2005) with four occupational therapists, four physiotherapists and four speech and language therapists. The pre-test sample consisted of two males and ten females with work experience ranging from 0 to 21 years. The levels of education varied, including vocational education, bachelor’s degree and master’s degree. Written informed consent was obtained from each participant. The subjects were asked to complete the DTI-G and to think aloud while responding (Willis, 2005). The verbal reports were audiotaped, transcribed and analysed according to the methods of qualitative content analysis (Mayring, 2010). Following the analysis of the pretest data, the wording of a number of items was adapted (see the Results section).

After creating an online version of the DTI-G with the open-source program LimeSurvey (LimeSurvey GmbH, Hamburg, Germany), physiotherapists and speech and language therapists were contacted by e-mail and asked to complete the DTI-G. Additionally, the link to the online questionnaire was published on the homepages of professional associations. Four universities were contacted, which forwarded the link on to their students. After completing the questionnaire, participants were asked if they would be willing to participate in the retest study after four weeks.

An exploratory factor analysis (EFA) using the principle components extraction and orthogonal varimax rotation was performed on the 41 items to explore the instrument’s factorial structure (Bühner, 2011). We decided to use principal components analysis based on the assumption that items are measured without error. In this case, principal components analysis provides the most parsimonious description of the data. Factors were extracted if their eigenvalue was >1. Oblique rotations were used when assumptions with regard to the factorial structure were not met. Items were deleted if their factor loading was below 0.30.

To examine the discriminatory power of each single question/item, the item discrimination index (IDI) was calculated. This index indicates the extent to which a single item measures the construct. Items were deleted if the IDI was below 0.20 (Bühner, 2011). Furthermore, face validity of questionable items was assessed within the research team based on logical and professional considerations.

Internal consistency of the DTI-G was assessed by calculating Cronbach’s alpha, alpha if item deleted, item-scale correlations, and item difficulty. Cronbach’s alpha indicates if individual items measure the same concept (Bühner, 2011).

To assess four-week test–retest reliability, the two-way random intraclass-correlation coefficient (ICC)_2,1 was calculated for the total score of the DTI-G. A two-way random model was chosen because raters were chosen randomly and answered each of the items (de Vet et al., 2011).

Additionally, known-groups validity was tested based on the duration of clinical experience. Subjects were divided into two groups using the median split method. Group differences were tested using the independent t-test.

After completion of data collection, data were analysed using the Statistical Package for the Social Sciences (SPSS version 21; IBM Inc., New York, USA). The two-sided significance level was set at a = 0.05.

The twelve interviews with physiotherapists, speech and language therapists and occupational therapists showed differences in the diagnostic process and thinking between the professionals. All four occupational therapists had difficulties to answer questions about diagnosis, differential diagnosis or procedure. They explained that occupational therapists basically have a different diagnostic process compared to other health professionals and that the focus is rather on activity than on symptoms and pathology. This finding indicates that two different versions of the questionnaire are needed: one version for speech and language therapists and physiotherapists and another version for occupational therapists; the latter was not within scope of this study. Therefore, occupational therapists were not included in the further validation process.

Generally, analyses of the eight interviews with physiotherapists and speech and language therapists showed that the majority of the questions were understandable. Ten of the 41 items had to be revised because the wording was not comprehensible. Questions were reworded to reflect a more practice-oriented language (see Appendix 1 for the translation of the DTI-G).

The link to the online version of the DTI-G was sent to approximately 10,000 speech and language therapists and 30,000 physiotherapists in Germany. In all, 800 (»2%) questionnaires were returned. A sample consisting of n = 388 physiotherapists (58% females) and n=122 speech and language therapists (88% females) completed the whole questionnaire. Participating physiotherapists had a mean age of 37.23 years (SD = 11.07) and a mean clinical experience of 12.91 years (SD = 9.87). Speech and language therapists had a mean age of 31.53 years (SD = 11.36) and a mean clinical experience of 6.06 years (SD = 8.34). After four weeks, 118 physiotherapists and 54 speech and languages therapists completed the questionnaire again. As the number of questionnaires returned by speech and language therapists was too low, only questionnaires of physiotherapists were included in the EFA. According to Bühner (2011), a good sample size for EFA should exceed 300.

A principle component factor analysis (PCA) was used to determine the underlying constructs of the scale. The Kaiser–Meyer–Olkin measure of sampling adequacy was 0.89, which suggests that the data were suitable for factor analysis. Bartlett’s test of sphericity reached statistical significance (p < 0.001), which supported the factor ability of the correlation matrix.

A PCA with promax oblique rotation was carried out for 36 items. Oblique rotations are recommended when assumptions in regard to the factorial structure are not made (Bühner, 2011). The PCA resulted in a five-factor model. A factor loading of 0.30 was used as the cutoff score for selecting the items (Bühner, 2011). Thus, items in this study that had a factor loading of 0.30 or higher were retained (Table 2). All other items, except items 6 and 16, were removed, resulting in remaining 21 items. Items 6 and 16 were considered substantial to the overall construct and were retained, although factor loadings were > 0.20 and < 0.30.

A PCA with promax oblique rotation with the remaining 21 items resulted in a five-factor solution that explained 55% of the total variance across all items. Factor 1 accounted for 29% of the total variance, factor 2 accounted for 9% of the total variance, factor 3 accounted

for 5% of the total variance, factor 4 accounted for 6% of the total variance and factor 5 accounted for 6% of the total variance (table 2).

Factors were given names that reflect the meaning of the respective items (see Appendix 2 for the translated items of the final version of the DTI-G). The labels of factors 1 and 2 were selected based on findings of Groves et al. (2003), and labels of factor 3 and 4 were based on those used by Bordage et al. (1990). Factor 5 was labelled based on consensus within the research team. Factor 1 was labelled “Recognition of meaningful information”; factor 2 “Information processing”, factor 3 “Flexibility in thinking”, factor 4 “Structure in knowledge” and factor 5 “Memory retention”.

A confirmatory principal component analysis with oblique rotation was carried out with the retest sample (n=176 consisting of 118 physiotherapists and 58 speech and language therapists). The maximum likelihood factor analysis with oblique rotation was used to test the goodness of fit of the five-factor model. The results, including the loading of each item on the factors, the correlation between the five factors, and the CFA model fit parameters, are presented in Figure 1. The results show an acceptable to good overall fit (CFI = 0.93; RMSEA = 0.05; SRMR = 0.06) of the model (Figure 1) (Hu & Bentler, 1999; Schermelleh-Engel, Moosbrugger & Müller, 2003).

Figure 1

The internal consistency of the DTI-G 5-factor model with 21 items was considered as good with Cronbach’s a = 0.84.

The four-week test–retest reliability was found to be good for the 21-item DTI-G with an ICC_2,1 = 0.87 (95% CI 0.81–0.91), p < 0.001 and n = 118 physiotherapists.

Additionally, known-groups validity of the 21-item DTI-G was tested. Following median split, the difference between participants with equal or more than 9 years of clinical experience (DTI = 97.90, SD = 13.83) and those with less than 9 years of clinical experience (DTI = 88.55, SD = 16.81) was significant (t₃₈₅ = 6.00, p < 0.001, 95% CI 7.82–10.88).

The internal consistency was good for the overall score (Cronbach’s a = 0.84) corresponding with the findings from other studies (Bordage et al. 1990, Jones 1997, Gelhar et al. 2014).

Another aim of our study was to investigate factorial structure of the DTI. However, the two-factor structure (flexibility in thinking and structure in memory) of the original DTI could not be confirmed. Our findings indicate a five-factor structure. To date, studies investigating the factorial structure of the DTI could not be detected; thus, comparison to the results of other studies is not possible.

The process of the translation and cross-cultural adaption was performed according to international recommendations (Beaton et al. 2000) The back translation was authorised by Bordage, the original author of the DTI. This approach guarantees that content validity of the inventory was maintained within the translation process. The cognitive interviewing technique, which was used in the pretest of our study, additionally supports content validity of the DTI-G.

The DTI-G consists of 21 items, which might be more practical and time-efficient for the use in future research projects than the original version with 41 items. However, removing 20 items from the original questionnaire might raise the question if the German version still measures the same construct. Two points need to be taken in account in regard to this question. First, to our knowledge, IDIs and factorial structure of the original English DTI have never been analysed. Doing so could also result in reduction of items. Second, many questionnaires have short forms that have been shown to measure the same construct as the long version. To strengthen construct validity, future studies should investigate concurrent validity of the DTI-G by comparing it to other instruments known to measure diagnostic ability.

Furthermore, the retest sample completed the 41-item DTI-G after four weeks. As we calculated the ICC for the 21-item version, this measure could be biased. Another reliability study with the 21-item instrument should be carried out.

The DTI-G is a reliable and valid inventory for use with German physiotherapists. It can be used to evaluate teaching concepts and improve diagnostic thinking in physiotherapy. This study focused on the validation of the DTI-G only for physiotherapists. Thus, the instrument should not be used for research in other professions.

Further research is necessary to evaluate convergent and divergent validity and validate the DTI-G for other health professions. In addition, sensitivity to change in the DTI-G needs to be explored. Further studies will be necessary to validate the DTI-G for other health professionals such as speech and language therapists, occupational therapists and physicians.

In addition, future studies should be conducted to generate norm data, in order to interpret the results of individual subjects.