New Page 1

2. Discuss validity and reliability of diagnosis

Diagnosis

  • Diagnosis within abnormal psychology means identifying and classifying abnormal behavior on the basis of symptoms, the patients’ self-reports, observations, clinical tests or other factors such as information from relatives.
  • Clinicians use psychological assessment and diagnostic manuals to make diagnosis. The diagnostic manuals help to classify and standardize diagnosis.
  • Diagnosis involves matching the results of the psychological assessment with classification systems such as DSM-IV-TR and ICD-10. The purpose of diagnosis is to find a treatment for the patient and to make a prognosis.

Diagnostic manuals

  • DSM-IV, now in its fourth revised version, is developed by the American Psychiatric Association. The manual lists what it terms “mental disorders”. For each of the 300 disorders there is a list of symptoms that the clinician could look for in order to diagnose correctly. A new fifth version is on its way. The diagnostic manual does not identify causes of psychological disorders (etiology) but merely describes symptoms.
  • ICD-10 (The International Classification of Diseases) is published by WHO (World Health Organization). The manual uses the term “mental disorder”. The diagnostic manual includes reference to causes of the disorders (etiology).

Reliability of diagnosis

  • Reliability in diagnosis means that clinicians should be able to reach the same correct diagnosis consistently if they use the same diagnostic procedure (e.g. standardized clinical interview, observation of the patient’s symptoms, neuropsychological examination with scanners and diagnostic manuals). This is called inter-judge reliability.
  • Reliability can be improved if clinicians use standardized clinical interview schedules, which define and specify sets of symptoms to look for. The individual psychiatrist must still make a subjective interpretation of the severity of the patient’s symptoms.
  • The introduction of diagnostic manuals has increased reliability of diagnosis over the year even though the manuals are not without flaws.
  • Reliability of diagnosis is a necessary prerequisite for validity. Rosenhahn (1973) performed a classic study that challenged reliability and validity of psychiatric diagnosis and showed the consequences of being labeled as “insane”. In this study eight pseudo-patients were diagnosed as suffering from severe psychological disorders but they were in reality imposters.

 Study: Cooper et al. (1972) The US-UK Diagnostic Project



  • The aim of the study was to investigate reliability of diagnosis of depression and schizophrenia.
  • The researchers asked American and British psychiatrists to diagnose patients by watching a number of videotaped clinical interviews.
  • The British psychiatrists diagnosed the patients in the interview to be clinically depressed twice as often. The American psychiatrists diagnosed the same patients to be suffering from schizophrenia twice as often.
  • The results indicated that the same cases did not result in similar diagnosis in the two countries. This points towards problems of reliability as well as cultural differences in interpretation of symptoms and thus in diagnosis.

Validity of diagnosis

  • Validity of diagnosis refers to receiving the correct diagnosis. This should result in the correct treatment and a prognosis (predictive validity). Validity presupposes reliability of diagnosis.
  • It is much more difficult to provide a correct diagnosis and give a prognosis for a psychological disorder than for a physical disorder because it is not possible to observe objective signs of the disorder in the same way.
  • The DSM-IV manual does not include etiology but only symptoms. Sometimes patients have symptoms that relate to different psychological disorders so it can be difficult to make a valid diagnosis.

Study: Mitchel et al. (2009) Meta-analysis of validity of diagnosis of depression

  • The study used data from 41 clinical trials (with 50,000 patients) that had used semi-structured interviews to assess depression.
  • The general practitioners (GPs) had 80% reliability in identifying healthy individuals and 50% reliability in diagnosis of depression. Many GPs had problems making a correct diagnosis for depression.
  • Generally GPs were more likely to identify false positive signs of depression after the first consultation. Michel et al. argued that GPs should see patients at least twice before making a diagnosis since accuracy of diagnosis was improved in studies that used several examinations over an extended period.
  • Evaluation of the study: (1) The strengths of meta-analysis are that it can combine data from many studies and it is possible to generalize to a larger population; (2) Limitations of meta-analysis are that it may suffer from the problem of publication bias; since data from many different studies are used there may also be problems of interpretation of the data because it is not certain that each study uses exactly the same definitions.

Rosenhan (1973) On being sane in insane places

Aim:

To test reliability and validity of diagnosis in a natural setting. Rosenhan wanted to see if psychiatrists could distinguish between “abnormal” and “normal” behavior.

Procedure:

This was a covert participant observation with eight participants consisting of five men and three women (including Rosenhan himself). Their task was to follow the same instructions and present themselves in 12 psychiatric hospitals in the USA.

Results:

  • All participants were admitted to various psychiatric wards and all but one were diagnosed with schizophrenia. The last one was diagnosed with manic depression.
  • All pseudo-patients behaved normally while they were hospitalized because they were told that they would only get out if the staff perceived them to be well enough.
  • The pseudo-patients took notes when they were hospitalized but this was interpreted as a symptom of their illness by the staff. It took between 7 to 52 days before the participants were released. They came out with a diagnosis (schizophrenia in remission) so they were “labeled”.
  • A follow-up study was done later where the staff at a specific psychiatric hospital were told that impostors would present themselves at the hospital and that they should try to rate each patient whether he or she was an impostor. Of the 193 patients, 41 were clearly identified as impostors by at least one member of the staff, 23 were suspected to be impostors by one psychiatrist, and 19 were suspected by one psychiatrist and one staff member. There were no impostors.

Evaluation

  • This controversial study was conducted nearly 40 years ago but it had an enormous impact in psychiatry. It sparked off a discussion and revision of diagnostic procedures as well as discussion of the consequences of diagnosis for patients. The development of diagnostic manuals has increased reliability and validity of diagnosis although the diagnostic tools are not without flaws.
  • The method used raises ethical issues (the staff were not told about the research) but it was justified since the results provided evidence of problems in diagnosis which could benefit others. There were serious ethical issues in the follow-up study since the staff thought that impostors would present, but they were real patients and may not have had the treatment that they needed.