Diagnostic interviews, the primary method for identifying substance use and mental disorders like depression and anxiety, exhibit varying levels of reliability depending on the specific condition, according to a study published in Jama Network Open.
Laura Duncan, a psychiatry professor at McMaster University in Ontario and one of the study’s authors, remarked that while diagnostic interviews are often seen as the “gold standard” for assessing mental disorders in clinical settings and research, they don’t always provide a “definitive benchmark” for validity and reliability.
Despite a long history of mixed evidence regarding their reliability, Duncan noted that these interviews are still widely regarded as the best option currently available, likely due to the absence of superior alternatives. The review examines data on the “test-retest reliability” of diagnostic interviews conducted between February 2024 and September 2025.
To evaluate the reliability of various mental health diagnoses, the authors used Cohen’s kappa coefficient. This method calculated how consistently patients received the same diagnosis when subjected to the same interview twice, accounting for the element of chance.
Overall, the reliability tended to be better for substance use disorders, peaking for opioid use disorder. Duncan explained that this is largely because the criteria for substance use disorders focus more on observable behaviors. For example, tracking the number of drinks consumed over a week is often simpler than recalling how many days one felt anxious or sad.
Dr. Michael First, a psychiatrist at Columbia University and creator of the Structured Clinical Interview for DSM-5 Disorders (SCID), expressed some frustration with the study. While he agreed that reliability issues exist and that diagnostic interviews sometimes fail to accurately diagnose individuals, he wished for more detailed information regarding which specific tools proved most reliable.
“It would be helpful to refer to this paper and say: ‘Oh, I should use this one for this reason.’ That would benefit the field,” he said, noting the lack of essential data in the study. Duncan responded that the available research during the study period was limited.
The review included studies on diagnostic tools such as the SCID and the Mini International Neuropsychiatric Interview (Mini), both designed to evaluate multiple mental health conditions. It also covered specific tools like the Clinically Administered PTSD Scale (Caps).
First critiqued the study for combining “fully structured” interviews with “semi-structured” ones. He explained that fully structured interviews tend to give more consistent results because they follow a strict format with no deviations allowed, even if a response is contradictory.
“If a person provides a contradictory answer, you can’t even mention it,” he emphasized. Such interviews are often used in large-scale epidemiological research and meant for administrators with minimal training.
In contrast, semi-structured interviews, intended for trained professionals, allow clinicians to adapt their questions as necessary. This flexibility can lead to more precise diagnostics but might also result in variations in patient responses across sessions.
While Duncan acknowledged the validity of First’s points, she stated that data to address these concerns is still lacking. The study aimed to extract information on interview formats but often found this information unclear or unreported. This deficit highlights a broader need for more rigorous approaches in psychiatric diagnostics.
First, despite his work on structured interviews, readily acknowledged their limitations. He articulated the ongoing hope among psychiatrists for the development of more objective laboratory tests for mental health conditions.
“We’ve been saying that for 50 years,” he noted. Duncan suggested that a future approach may involve clinicians moving away from rigid diagnostic categories to consider symptoms on a spectrum or continuum.
