Applying Research To Practice: A Guide For Clinicians

Applying Research To Practice:
A Guide For Clinicians

Sam Goldstein, Ph.D.<!-- This is a SamGoldstein.com Monthly Article - April, 2000
Copyright ? 2000 Dr. Sam Goldstein - All Rights Reserved

Increasingly, clinicians recognize the importance of maintaining ties to scientific research as part of their vocational activities. The field of psychology has literally been bombarded with new journals and a significant increase in peer reviewed publications. Even the most dedicated clinicians have difficulty keeping up with this literature in all but one or two areas of interest. The phenomenon of increased literature has been especially prevalent in the field of ADHD. Literally thousands of articles were published on the subject between 1974 and 1994 (Resnick and McEvoy, 1994). Since that time the monthly rate of articles published on the subject of ADHD continues to increase at a near light speed pace. Many newsletters and an entire peer-reviewed journal (Journal of Attention Disorders) are devoted to ADHD. The articles in these journals are read by many clinicians, often with the goal of gleaning some bit of clinical utility. It is easy, unfortunately, to fall into one of the many interpretation traps when attempting to apply research generated information to clinical practice. In this article, I will attempt to assist clinicians to gain a better understanding of the process of applying research to practice by offering an example of three recently published articles and suggesting a number of guidelines.

A recent issue of the Journal of the American Academy of Child and Adolescent Psychiatry included three articles, each of which spoke in part to the future psychiatric and related risks for children receiving diagnoses of ADHD or hyperactivity (Biederman, Faraone, Milberger, Jetton, Chen, Mick, Greene and Russell, 1996; Taylor, Chadwick, Heptinstall and Danckarets, 1996; MacDonald and Achenbach, 1996). All three of these articles share a common objective in attempting to assess the present and future risk of children with ADHD or hyperactivity developing more severe disruptive disorders. Individually, they also speak to the relationship between oppositional defiant disorder (ODD) and conduct disorder (CD) as well as the relative risk of non-disruptive disorders developing in children with ADHD. Each of these studies represent small parts of long- term projects from three well-respected research groups, two in the United States and a third (Taylor, et. al.) in Great Britain. There are, however, a number of significant differences in subject selection and methodologies which make comparison of these studies somewhat tenuous. Yet, they pose many of the same questions and most clinical readers will usually skim the first parts of articles such as these, settling on the Discussion sections to glean a bit of valuable information. For these three articles, this process exemplifies the dilemma of moving from research to practice.

Biederman, et. al. examined 140 children with ADHD and 120 normals at baseline and four years later. Of the ADHD subjects at baseline, 65% experienced comorbid ODD and 22% CD. Among those with ODD, 32% also had comorbid CD. All but one child with CD also had ODD that proceeded the onset of CD by several years. The group with all three diagnoses, not surprisingly, had more severe symptoms and a wider range of symptoms than children with ADHD alone or ADHD plus ODD. ODD without CD at baseline in this population did not increase the risk for CD at the four year follow-up by mid- adolescence. Based on factor analysis the authors defined two subtypes of ODD associated with ADHD. One type appeared prodromal to CD, the other sub-syndromal to CD and not likely to lead to the progression of CD in later years. The ODD subtypes had different correlates, course and outcome, suggesting a clinically valuable basis for evaluating future risk in some ADHD children. The ADHD only children had slightly more risk of major depression, bipolar and anxiety disorders than controls. They had less risk than those with the combined, disruptive disorders. Many clinicians reading this article immediately recognize that these results parallel and are supported by other research studies reflecting comorbid risks for ADHD.

Taylor, et. al. completed a follow-up study of children identified by parent and teacher ratings in a large community survey of six to seven year olds, as demonstrating pervasive hyperactivity, conduct problems, a mixture of both problems or neither. The children were later investigated at ages sixteen to eighteen years. Hyperactivity was deemed a risk factor for later developmental problems, even allowing for the co-existence of conduct problems. The authors concluded that the sequelae of hyperactivity included much greater risk for developing psychiatric disorders, persisting hyperactivity, social and peer problems, even violence and anti-social behavior. The data from this article suggests that there is a developmental pathway through which hyperactivity acts as a risk factor, increasing the likelihood of impaired social development, leading to additional psychiatric disorders independent of the existence of conduct problems.

MacDonald and Achenbach tested the extent to which an empirically derived factor of attention problems and the continuation of early comorbid conduct problems accounted for poor outcome scores on the Child Behavior Checklist and related measures three and six years after initial assessment. A nationally representative U.S. sample was assessed three times over six years. Children deviant on both attention and conduct problem scales scored significantly higher on behavior problems at outcome than did those deviant on only attention or conduct problems. After controlling for initial conduct problems, initial attention problems made little unique contribution to later conduct problems. Predictive patterns were similar across gender and age groups.

MacDonald and Achenbach conclude that "children who present with attention problems without conduct problems have a relatively low risk of developing severe and persistent anti-social behavior, at least through age twenty-two years" (pg. 1237). Taylor, et. al. conclude that pervasive hyperactivity is a "risk factor for later development, even allowing for the co-existence of conduct problems" (pg. 1225). Finally, Biederman, et. al. conclude and provide supporting related research to suggest that although ADHD may be a risk factor for later disruptive problems, it is the comorbidity of ODD and the development of CD in the ADHD population that predicts the greatest risk.

What information should clinicians take away from these three articles that may be helpful in clinical practice? Are these three studies equivalent? Should they be used to support, at least in partial ways, each other? Is it fair to conclude that ADHD is a risk factor for present and future development of disruptive and non-disruptive disorders, perhaps acting as a catalyst, increasing the risk of future problems when it presents with ODD and CD. This appears to be a middle of the road conclusion from these three articles. Should clinicians inform parents that children with early histories of "hyperactivity" have horrific future outcome or should they tell parents that despite high scores for the inattention factor on the Child Behavior Checklist, in the absence of other elevated scores, there is little risk of future psychiatric problems? Even the knowledgeable reader is left with unanswered questions.

Perhaps to address this issue clinicians should cease reading peer reviewed research but wait for others to review and integrate findings and publish them in texts (e.g., Goldstein and Goldstein, 1990; Goldstein, 1997; Barkley, 1990). I for one would not suggest that clinicians adopt this policy. This solution will place their knowledge base consistently a number of years behind the available research. My texts and others, however, play a valuable role in coordinating research and placing it at clinicians' fingertips. I would instead suggest that clinicians continue and in fact increase their reading of current research, but do so with an understanding of the limitations of applying what is read in research to clinical practice. With this in mind I would like to suggest a number of guidelines:

1. Read the entire article. Although time is often limited and it is attractive to read the Abstract and jump to the Discussion section, this is an inefficient and ineffective way of reviewing published research.
2. Know something about what you are reading. Introduction sections of peer reviewed articles attempt to set a foundation for what is known and what needs to be known. If you know little about the subject about which you are reading, it will be difficult to determine whether the authors are providing a fair and thorough literature foundation. All three of the articles just reviewed provided very thorough and comprehensive literature reviews in their introductions. All three, however, spoke to literature in part that did not directly apply to their studies. Taylor, et. al., for example, cited ADHD outcome research when in fact their hyperactive group was chosen by very different selection and definition criteria. MacDonald and Achenbach also in part spoke to ADHD outcome data based on subjects chosen categorically rather than based on empirical dimensions as done in the Child Behavior Checklist.
3. Read the Method section carefully. Although the method of subject selection, procedure and statistical analyses are critically important, they do not appear to be of much interest to most readers. For example, in the Journal of the American Academy of Child and Adolescent Psychiatry, the Method sections recently have been printed in type half the size of the full article. It is now not only tedious but uncomfortable to read this section.
4. Evaluate subject selection. Most readers assume that once selected groups are given a label (e.g., ADHD), they are somehow equated. Especially in the field of ADHD this has been a problem. As the categorical definition of ADHD changes, comparisons of research articles published over the years becomes nearly impossible. For the three articles reviewed, one group of children was chosen based upon a list of symptoms, the second based upon categorical diagnosis and the third, based upon an empirically derived, set of scales. The Taylor, et. al. subject group was referred to as hyperactive, the Biederman, et. al. group as ADHD and the MacDonald and Achenbach group as attention problems. Readers with a critical eye will quickly recognize that these three groups of children are not identical.
5. Understand basic statistics. Although it is not necessary to possess a working understanding of all complex statistical procedures employed today, it is necessary for clinicians to understand basic, statistical processes. For example, the fact that a particular task may be sensitive to identifying children with ADHD does not mean it should be immediately put into clinical practice. Clinicians often misinterpret that a test that is sensitive because it discriminates between two pre-determined groups of children (e.g., ADHD versus normals) will have clinical utility. But in clinical practice, one starts with a large group of undiagnosed children experiencing a specific symptom and attempts to determine if in that population the symptom has positive predictive power. Many of the proposed tests to screen for ADHD may be sensitive but have either poor positive or negative predictive power. These include subtest analysis from the Wechsler as well as continuous performance tests (Golden, 1996).
   Among the many issues that clinicians also need to understand is that correlation does not imply causation. Simply because two events occur beyond a chance level does not imply that one is causative of the other. Clinicians need to be careful not to take away interpretations of causation from even the best controlled research studies. In the three studies reviewed, although ADHD appears to be risk factor, it's relationship to outcome is correlational not necessarily causative.
6. Start at the end of the Discussion section. Most articles conclude with words of caution as to what conclusions can or cannot be drawn from the data presented. These cautions should be read carefully and well understood. They should be kept in mind when the remainder of the Discussion section is read.
7. Take three. I always attempt to take three ideas, conclusions, questions or a combination thereof away from each article I read. As attractive as it may be, I also attempt to be very careful to not build an entire theory or foundation for clinical recommendations or decisions based upon a single citation. In the field of psychology, the use of case example or a single study to justify clinical practice or recommendations (e.g., a single study suggests EEG biofeedback may impact ADHD so the clinician now recommends everyone receive this treatment) has and unfortunately will continue to be done.

Clinicians are increasingly faced with the arduous task of not only keeping up by reading research but evaluating that research and integrating what is learned into clinical practice. By understanding the processes of research, the means of interpreting methodology and statistics, by possessing a basic knowledge of the research being read and by drawing cautious conclusions, clinicians can make reasoned and reasonable decisions concerning the impact research has upon their clinical practice.