Executive Summary

Psychotic disorders have a lower prevalence than other forms of mental illness such as depression and anxiety disorders, yet they impose a considerable public health burden because of their impact on sufferers and their families (Keith, Regier, & Rae, 1991). Persons with psychotic disorders also utilise a disproportionately high segment of health services.

Valid and reliable assessment of any disorder is a necessary precursor to effective treatment. Lengthy interview instruments exist for the assessment of psychotic disorders, but they often require accredited training to administer, and their length means they may not be appropriate for all situations. Validated screening instruments provide a useful alternative to the full assessment of a disorder. They have been developed for the assessment of mental disorders such as depression (the Beck Depression Inventory; Beck, Ward, & Mendelson, 1961) and anxiety (the State-Trait Anxiety Inventory; Spielberger, 1983). However, there has been a lack of effective, validated instruments for screening individuals for psychotic illness.

The aim of this study was to examine the validity of a 7-item Psychosis Screener (PS) compared to full diagnoses of psychotic disorders using clinician ratings (ICD-9 classification) and derived from the Diagnostic Interview for Psychosis (DIP) (ICD-10 and DSM-III-R). The Psychosis Screener (PS) uses elements of the Composite International Diagnostic Interview (CIDI) to assess the presence of characteristic psychotic symptoms. The Psychosis Screener comprises 7 items, three of which are asked only if the respondent endorses a previous question. The first 6 items cover the following features of psychotic disorders: delusions of control, thought interference and passivity (Question 1 and 1a); delusions of reference or persecution (Question 2 and 2a); and grandiose delusions (Question 3 and 3a). The final item records whether a respondent reports ever receiving a diagnosis of schizophrenia.

Narrow and broad definitions of psychosis were used: the narrow definition of psychosis was limited to diagnoses of either schizophrenia or a schizoaffective disorder; and the broad definition of psychosis included diagnoses of affective psychoses in addition to schizophrenia and schizoaffective disorder. Receiver operating characteristic (ROC) analyses were conducted using data from two samples: the first (n=87) contained persons receiving inpatient treatment in Perth, Western Australia (WA); and the second (n=259) was drawn from the WA Study of Low Prevalence (Psychotic) Disorders.

Two definitions of psychosis were used in the ROC analyses, and these affected the findings quite markedly. The broad definition of psychosis classed schizophrenia, schizoaffective disorder, and affective psychosis as psychotic disorders. When this broad definition of psychosis was used with ICD-9 diagnoses as the standard (in sample 1), the screener did not fare better than chance. However, this may have been related to the fact that diagnoses for sample 1 were obtained from clinical records which are coded using ICD-9-CM codes. This may have lead to some incorrect categorisation of patients as cases due to discrepancies between ICD-9 and ICD-9-CM codes, particularly for affective psychoses. This possibility is supported by the finding that when using two other diagnostic systems as ‘gold standards’, the screener was able to discriminate adequately between cases and non-cases, as assessed by the area under the ROC curve (the AUC). For both ICD-10 and DSM-III-R diagnostic systems (using sample 2), the optimal cut-off point was zero, indicating that a score of 1 or more on the screener indicated a case according to this definition of psychosis.

Using the narrow definition of psychosis, in which only those with a diagnosis of schizophrenia or schizoaffective disorder were classified as cases, the screener was well able to discriminate between cases and non-cases using any of the three diagnostic systems as the standard. A score of three or more on the screener was the optimal score for indicating a case for all three ‘gold standards’.

The analyses carried out indicated that the psychosis screener developed as a brief screening instrument for the presence of psychosis has a moderate ability to discriminate between those who meet diagnostic criteria for psychotic disorders, and those who do not. This represents an advance in efforts to develop a measure that will be an effective screen for these low prevalence disorders. Consideration must be given to the nature of the population with which a screening test is to be used before a cut-off point is selected.

Citation: Degenhardt, L., Hall, W., Korten, A. and Jablensky, A. (2005) Use of brief screening instrument for psychosis: Results of a ROC analysis, Sydney: National Drug and Alcohol Research Centre.