Newly updated, this is a comprehensive guide to ODD and conduct disorder (CD) in children aged 3-14 for professionals, students, and researchers. * Summarizes the most important empirical knowledge across a broad array of topics, with a focus on the latest research and meta-analyses, as well as high-quality older studies * Includes revised diagnostic conceptualizations for ODD and CD from DSM-V and the upcoming ICD-11 classification systems, with particular attention to similarities, differences, and information about an angry-irritable subtype for ODD * Provides updated reviews of biological and social-cognitive risk and protective factors and the evidence base for relevant treatment and prevention procedures * Describes best practices for assessment, treatment, and prevention for children and their families, based on the clinical and research work of the well-respected author team
Ebooka przeczytasz w aplikacjach Legimi na:
Liczba stron: 636
About the Authors
1 Behaviors and Disorders
Socially Appropriate Behaviors
Concluding Summary Points
2 Developmental Perspectives
Development and Developmental Psychopathology
Development of Behavior and Processes over Time
Development in Problematic Externalizing Behaviors
Developmental Models and Etiology of Disorder
Concluding Summary Points
3 Basic Issues on Etiology
The Brain–Mind–Environment Relationship
Risk Factors and the Causation of Disorders
The Comorbidity Between ODD or CD and ADHD
An Etiological Framework
Concluding Summary Points
4 Individual Characteristics
Behavioral and Molecular Genetics
Minimal Physical Anomalies
Intelligence and Language Development
Social Information Processing and Social Problem‐Solving
Anger, Anger Regulation and Emotional Regulation
Concluding Summary Points
5 Environmental Characteristics
Effect of the Environment on Individual Biological Characteristics
Contextual Family Factors
Contextual Peer Factors
Contextual Community and School Factors
Environmental Factors Affect Social Information Processing
Concluding Summary Points
6 Clinical Assessment
An Overview of the Assessment Procedure
Initial Interview with the Parents and the Child
Hypothesis on Possible Diagnoses and Comorbidities
Interview and Observation of the Child
DSM‐5 Orientated Interview with the Parents or the Child
Categorical l Diagnosis, Diagnostic Formulation, and the Treatment Plan
Discussion of Diagnosis and Treatment Plan with the Parents, and Use of Psychoeducation in Order to Improve Treatment Engagement
Concluding Summary Points
7 Intervention and Intervention Development
The Intervention Continuum
Developmental Models and Intervention Development
Evidence‐Based Interventions (EBIs)
Concluding Summary Points
8 Behavioral Parent Training
General Characteristics of Programs and Effectiveness
Examples of Programs
Concluding Summary Points
9 Cognitive Behavioral Therapy
Roots in Observational Learning, Social Learning, and Developmental Theories
Early Forms of CBTs and Theories with Children
Examples of Intervention Programs for Conduct Problem Behavior
Intervention in Preschool and Early Childhood
Intervention in Later Childhood
Intervention in Early Adolescence
Concluding Summary Points
Concluding Summary Points
11 Multicomponent Intervention
Need for Multiple Components
Intervention in Early Childhood
Intervention in Later Childhood
Intervention in Early Adolescence
Concluding Summary Points
12 Factors which Influence Intervention Delivery and Outcomes
Dissemination to Real‐World Settings
At the Intervention Program Level
At the Client Level
At an Intervention Group Level
At the Level of the Relationship Between the Clinician and the Child or the Parent
At the Level of the Clinicians and Their Practice Settings
At the Level of Cost‐Effectiveness of the Intervention Efforts
Concluding Summary Points
End User License Agreement
Table of Contents
Walter Matthys and John E. Lochman
This second edition first published 2017© 2017 John Wiley & Sons, Ltd
Edition history: John Wiley & Sons, Ltd (1e, 2010)
Registered OfficeJohn Wiley & Sons, Ltd, The Atrium, Southern Gate, Chichester, West Sussex, PO19 8SQ, UK
Editorial Offices350 Main Street, Malden, MA 02148‐5020, USA9600 Garsington Road, Oxford, OX4 2DQ, UKThe Atrium, Southern Gate, Chichester, West Sussex, PO19 8SQ, UK
For details of our global editorial offices, for customer services, and for information about how to apply for permission to reuse the copyright material in this book please see our website at www.wiley.com/wiley‐blackwell.
The right of Walter Matthys and John E. Lochman to be identified as the authors of this work has been asserted in accordance with the UK Copyright, Designs and Patents Act 1988.
All rights reserved. No part of this publication may be reproduced, stored in a retrieval system, or transmitted, in any form or by any means, electronic, mechanical, photocopying, recording or otherwise, except as permitted by the UK Copyright, Designs and Patents Act 1988, without the prior permission of the publisher.
Wiley also publishes its books in a variety of electronic formats. Some content that appears in print may not be available in electronic books.
Designations used by companies to distinguish their products are often claimed as trademarks. All brand names and product names used in this book are trade names, service marks, trademarks or registered trademarks of their respective owners. The publisher is not associated with any product or vendor mentioned in this book.
Limit of Liability/Disclaimer of Warranty: While the publisher and authors have used their best efforts in preparing this book, they make no representations or warranties with respect to the accuracy or completeness of the contents of this book and specifically disclaim any implied warranties of merchantability or fitness for a particular purpose. It is sold on the understanding that the publisher is not engaged in rendering professional services and neither the publisher nor the author shall be liable for damages arising herefrom. If professional advice or other expert assistance is required, the services of a competent professional should be sought.
Library of Congress Cataloging‐in‐Publication Data
Names: Matthys, Walter. | Lochman, John E. (John Edward), author.Title: Oppositional defiant disorder and conduct disorder in childhood / Walter Matthys and John E. Lochman.Description: Second edition. | Chichester, West Sussex ; New York, NY : John Wiley & Sons, Inc., 2017. | Includes bibliographical references and index.Identifiers: LCCN 2016031582 (print) | LCCN 2016042350 (ebook) | ISBN 9781118972564 (cloth) | ISBN 9781118972557 (pbk.) | ISBN 9781118972533 (pdf) | ISBN 9781118972540 (epub)Subjects: LCSH: Oppositional defiant disorder in children. | Conduct disorders in children.Classification: LCC RJ506.O66 M38 2017 (print) | LCC RJ506.O66 (ebook) | DDC 618.92/89142–dc23LC record available at https://lccn.loc.gov/2016031582
A catalogue record for this book is available from the British Library.
To Paula and Linda
Walter Matthys, MD, PhD, is Professor Emeritus at Utrecht University, the Netherlands. His clinical work as a child and adolescent psychiatrist was based at the Department of Child and Adolescent Psychiatry, University Medical Centre Utrecht. He is an associate editor of the Journal of Abnormal Child Psychology. His research focuses on neurocognitive and social cognitive characteristics in children with aggressive behavior, especially children with oppositional defiant disorder and conduct disorder. His research interests also include interventions to prevent and treat these disorders.
John E. Lochman, PhD, ABPP, is professor and Doddridge Saxon Chairholder in Clinical Psychology at the University of Alabama, where he also directs the Center for Prevention of Youth Behavior Problems. He is a past editor of the Journal of Abnormal Child Psychology; has received the Distinguished Career Award from the Society of Clinical Child and Adolescent Psychology; and has been president of the American Board for Clinical Child and Adolescent Psychology and of the Society for Children and Family Policy and Practice. His research interests include risk factors, social cognition, and intervention and prevention with aggressive children.
The world of psychopathology finds itself in a period of rapid change and considerable disagreement. On the one hand, given the considerable prevalence and major impairments linked to mental disorders, the clinical need has never been greater for evidence‐based assessments of—and sound, exportable, evidence‐based treatments for—all such conditions, especially the topic of interest for the present volume, oppositional defiant and conduct disorders in youth. In keeping with the urgency of the matter at hand, scientific efforts are increasing at a rapid pace, and attempts to disseminate validated treatments are at full strength. On the other hand, the prevalence of mental illness is, if anything, increasing rather than decreasing. Moreover, scientists continue to debate the validity of categorical designations of psychiatric conditions. Indeed, it is hard to find a month that goes by without renewed contention about the merits of official nosologies (such as the Diagnostic and Statistical Manual of Mental Disorders and International Classification of Diseases) and their highly questioned abilities to capture the dynamic processes related to the onset and maintenance of clinically significant emotional and behavioral disturbance. Alternative conceptions, such as the Research Domain Criteria (RDoC), which prioritize dimensions of both behavior patterns and underlying neural substrates, are vying for contention as means of best understanding the relevant mechanisms.
Nowhere is this tension more evident than in child and adolescent mental health, where the tender ages of the individuals under consideration magnify relevant clinical and ethical concerns. With regard to so‐called externalizing behavior patterns, integrative views must synthesize intra‐individual and environmental factors—that is, psychobiology and context—with an explicit focus on developmental principles and processes in order to emerge with greater comprehension of the processes by which young people come to display patterns of emotion dysregulation and executive dysfunction, propensities for violating the rights of others, and tendencies toward commission of violent acts that lead to clinical referral.
Thus, what a pleasant surprise—and indeed honor—to have had the opportunity to preview Oppositional Defiant Disorder and Conduct Disorder in Childhood, the second edition of the volume edited by Matthys and Lochman on oppositional defiant and conduct disorders in children. On second thought, I should have experienced no surprise at all, given the distinguished careers of these authors. Yet, the levels of concision, precision, and integration, as well as sorely needed clarity, in the pages that follow are unprecedented. The writing embraces a combination of elegance and prescription, emerging from their masterful synthesis of the relevant scientific and clinical literatures.
Indeed, this relatively compact book is undergirded by the strong presence of a developmental psychopathology (DP) framework. This focus includes explicit coverage of such essential principles as the mutual interplay between typical and atypical patterns of development, the need to consider multiple levels of analysis, the essential nature of patterns of continuity and transaction in understanding psychopathology, and the dynamic mechanisms involved in children’s externalizing behavior patterns, seldom captured in static diagnostic categories such as oppositional defiant disorder (ODD) or conduct disorder (CD). Embracing such principles, the book begins by providing succinct yet extremely informative coverage of the toddler and preschool years, setting in motion lifelong trajectories of disruption and impairment. The authors are careful to address the underlying psychobiological vulnerabilities relevant for those children considered “early starters” for antisocial behavior; and yet, at the same time, Matthys and Lochman completely embrace transactional models involving the roles of family, school, neighborhood, and cultural context in molding and shaping such early vulnerability. Readers will leave with a clear sense of the importance of such key issues as the age of onset of relevant symptoms, comorbid ADHD‐related behavior patterns, mediating cognitive mechanisms, parenting strategies, and peer rejection as essential perspectives linked to developmental continuities or discontinuities in the relevant behavior patterns.
Overall, the contents of the first half of the book comprise as potent an instantiation of the need to embrace core DP principles as I have ever seen in an accessible book on ODD and CD. Far different from most guidebooks on child mental health, these pages provide a lucid accounting of gene–environment interplay, setting the stage for the unfolding patterns of oppositional and emotionally negative behavior patterns plaguing all too many youth. These contents are a tour de force.
The second group of chapters addresses, head on, the kinds of evidence‐based assessment practices essential for accurate diagnosis as well as evidence‐based medication and psychosocial intervention strategies essential for treatment. The authors know, first hand, just what it takes to transcend quick‐and‐dirty diagnostic practices in order to gain the kinds of historical and norm‐based data required to understand the child and his or her relevant contexts—and to distinguish high but developmentally normative behavior patterns from clinical‐level conditions. Moreover, their exposition of intervention strategies is masterful, including exposition of the kinds of procedures that must be delivered not just to children but to parents and teachers (and communities). The authors emphasize both fidelity and flexibility in relation to treatment guides and manuals. Their prescriptions have implications that go far beyond the relatively small number pristine clinical trials conducted in the field (i.e., “efficacy” trials) by providing explicit focus on how to make things work in the real world of families, schools, and community settings (i.e., “effectiveness” research).
In the end, Oppositional Defiant Disorder and Conduct Disorder in Childhood will be of major value to basic scientists, clinical scientists, and practicing diagnosticians and clinicians everywhere who desire to understand the latest research findings, offering maximum guidance for bettering the lives of children and families. It provides a benchmark for how the best of science can be translated and applied to the urgent clinical needs of the unfortunately large numbers of children and families who struggle with oppositionality and frank aggression. Given the backdrop of a world that is increasingly integrated and global in its communication patterns, and in which propensities toward violence are increasingly embedded in clinical, school, legal, and police/enforcement‐based settings, the need for authoritative, sensitively presented information on etiology, maintenance, prevention, and treatment has never been stronger. This book therefore fills a major need.
Stephen P. Hinshaw,University of California, Berkeley;University of California, San Francisco
This book offers a brief but comprehensive overview of the empirical knowledge and associated clinical information regarding oppositional defiant disorder (ODD) and conduct disorder (CD) in children aged 3–14 years. Since the amount of research in this subject is vast, we have avoided presenting an extended review of the literature of some areas while neglecting others. Instead, we have given priority to conciseness and clarity in the presentation of a broad array of topics.
The book opens with an introductory chapter on relevant terms. We then present the developmental psychopathology perspective (Chapter 2). The section of the book on etiology starts with a chapter on basic issues (Chapter 3), followed by two detailed chapters on individual and environmental characteristics (Chapters 4 and 5). We then describe the clinical assessment from a decision‐making point of view (Chapter 6). The section on intervention opens with a chapter on general issues (Chapter 7), followed by chapters on behavioral parent training (Chapter 8), cognitive‐behavioral therapy (Chapter 9), pharmacotherapy (Chapter 10), and multicomponent interventions (Chapter 11). Finally, issues relating to the delivery of intervention are discussed in Chapter 12.
This synopsis is intended to be a guide for professionals and will be useful for students and researchers as well. Some information, for example, on neurobiological characteristics, will serve as background information for clinicians and will be relevant for their accurate general understanding of the initiation, development, and maintenance of the disorders. The background etiological chapters can also help clinicians in understanding subsets of the children they see. Students and beginning researchers will find a quick overview of the whole field, while advanced researchers may find essential information on topics that are not their primary focus of interest and expertise.
For decades, the research fields of ODD and CD (and aggressive and anti‐social behavior), on the one hand, and attention‐deficit/hyperactivity disorder (ADHD) (and hyperactive, impulsive, and inattentive behavior), on the other, have been split as if these two groups of disorder and problem behavior are independent of each other. In fact, however, they often co‐occur. The separation of research areas of ODD/CD and ADHD has hampered our understanding of these two groups of related disorders. Fortunately, the last decade has seen an integration of the research fields of these disorders. In this book, we have paid much attention to the relation between these disorders, with respect both to etiology, assessment, and treatment, and to neurobiological factors in ODD and CD, as information on this topic has grown rapidly over the last few years and is essential for an accurate understanding of these disorders.
In reviewing the literature, we pay particular attention to recently published studies and meta‐analyses, if available, without neglecting older, high‐quality studies. The selection of studies, of course, reflects our own views on the subject. Likewise, we give personal comments on issues. We include our clinical and research experience in the chapters on clinical assessment and interventions. To make the book accessible, we have included a conclusion in the form of summary points at the end of each chapter.
The first edition of this book was published in 2010. Revising and updating the book was needed for several reasons. First, diagnostic criteria for ODD and CD have changed since 2013 when the DSM‐5 was published. In addition, presently the ICD‐10 (1996) of the World Health Organization also is being revised. Second, information on the underlying neurobiological, psychological, and environmental factors has grown rapidly. Therefore, we updated the empirical research on these factors. Third, many intervention studies have been published the last few years. An update of our insight into the effectiveness of evidence‐based interventions was therefore needed.
We would like to thank the publisher Wiley‐Blackwell for the invitation to update the book, and the many people associated with the publisher for their assistance: Darren Reed, Roshna Mohan, Aravind Kannankara, and Vimali Joseph.
Walter MatthysJohn E. Lochman
All children sometimes refuse to comply. And a lot of children occasionally get involved in fights. Also, various children lie at times. Although these behaviors are inappropriate, from a clinical point of view, they need not be of great concern if they occur infrequently and in an isolated manner. However, when these behaviors occur in a cluster and repeatedly in a particular child, there is reason to be worried. In this chapter, we first look at the various types of socially inappropriate, disruptive behaviors that have been discerned, and then consider related clusters of behaviors or diagnostic categories that have been distinguished. Finally, we discuss appropriate behaviors that may be underdeveloped in children with maladjustment.
Oppositional behavior or noncompliance is behavior in which a child resists a caregiver. A range of oppositional behaviors may be discerned, from passive forms of noncompliance to active forms of noncompliance (Kochanska & Aksan, 1995). Thus, children may ignore a parental direction, which is an example of passive noncompliance, but children may also directly refuse a parental command, which is a form of mildly active noncompliance. In addition, children may angrily reject parental commands or prohibitions, which is a form of severe noncompliance or defiance. In preschool children, moreover, a distinction needs to be made between normative noncompliance and clinically significant noncompliance or oppositionality (Wakschlag & Danis, 2004). Normative noncompliance reflects the young child’s self‐assertion and is driven by the desire to do something autonomously (Wakschlag & Danis, 2004). Normative or self‐assertive noncompliance generally is short‐lived, whereas clinically significant noncompliance is more intransigent (Wakschlag & Danis, 2004).
Aggression is behavior deliberately aimed at harming people (Parke & Slaby, 1983). Hitting other children is an example of physical aggression. There are, however, other forms of aggression. Speech also may harm people, either as a possible precursor of physical aggression such as in verbal threats, or as a means to denigrate or provoke another child. For example, this occurs when children call each other names. Relational aggression is another form of aggressive behavior (Crick & Grotpeter, 1995). It is defined as damaging interpersonal relationships or feelings of inclusion. Malicious gossiping and threatening to withdraw friendship are examples of relational aggression. In this book, use of the term “aggression” implies physical or verbal aggression. When relational aggression is discussed, this is made explicit.
Among these various forms of aggressive behavior, distinction has been made between reactive and proactive aggression (for reviews, see Dodge, 1991; Kempes, Matthys, de Vries, & Van Engeland, 2005; Vitaro, Brendgen, & Barker, 2006). Reactive aggression is an impulsive aggressive response to a frustration, a perceived threat, or a provocation. On the other hand, proactive aggression is controlled aggressive behavior that anticipates a reward. Reactive aggression also has been called defensive or “hot‐blooded” aggression, whereas proactive aggression has been called instrumental or “cold‐blooded” aggression.
When considering aggression, one may distinguish differences in the underlying motivation (or the “whys” of aggressive behavior) from differences in the various forms of aggression (the “whats” of aggressive behavior) (Little, Jones, Henrich, & Hawley, 2003). Thus, the distinction between reactive and proactive aggression (the “whys”) may be applied both to physical, verbal, and relational aggression (the “whats”). One example of verbal reactive aggression in children is to get angry and swear at adults when corrected. One example of physical reactive aggression is to strike back when teased by a peer. To threaten another child in order to get his/her own way is an example of verbal proactive aggression. To incite other children to act against a child whom he or she dislikes is an example of proactive relational aggression. Although reactive aggression and proactive aggression are highly correlated, correlations drop dramatically after the distinction has been made between the form and the motivation of aggression (Polman, Orobio de Castro, Koops, van Boxtel, & Merk, 2007).
Antisocial behavior is defined as behavior by which basic norms, rights, and rules are violated. Thus, when children lie, they violate the norm of speaking the truth; when they steal, they violate the right of the protection of one’s property; and when they are truant, they violate a rule. “Antisocial behavior” is often used as a general term for the various inappropriate behaviors such as oppositional and aggressive behavior. When children repeatedly resist requests, instructions, or corrections given by adults, they indeed violate the norm to be obedient to their parents, elders, or teachers. Further, when children beat their peers, they violate their peers’ right of physical integrity.
When antisocial behaviors are legal violations, they are called delinquent behaviors. Depending on the age of the child, behaviors such as theft, running away, truancy from school, and setting fires are considered to be delinquent. Legislation among countries, and among states within countries, largely vary, so that the same behavior, for example, drinking alcohol, is considered as illegal in one country or state but not in another.
There is another term that is relevant here—“psychopathy.” Psychopathy refers to personality characteristics such as an absence of empathy, an absence of guilt, an absence of anxiety, shallow emotions, and the inability to form and sustain lasting relationships (Cleckley, 1976; Hare, 1993). Thus, psychopathy does not refer to a specific set of behaviors but to underlying characteristics of individuals.
The construct of psychopathy in adults consists of various dimensions. The several dimensions which have been found to be useful in children and adolescents are callous–unemotional traits (for a review, see Frick, Ray, Thornton, & Kahn, 2014) and narcissism (Barry et al., 2007). Callous–unemotional traits are characterized by lack of guilt and remorse, lack of concerns for the feelings of others, shallow or superficial expressions of emotions, and lack of concern regarding performance in important activities (Frick, 2009). Callous–unemotional traits constitute the affective factor of psychopathy and designate a particularly aggressive subgroup of children and adolescents with antisocial behavior (Frick et al., 2014). Callous–unemotional traits have been found to have moderate stability in longitudinal research (Barry, Dunlap, Lochman, & Wells, 2009; Pardini, Lochman, & Powell, 2007). Among children and adolescents with antisocial behavior, those with high levels of callous–unemotional traits display more instrumental aggression and show a more stable pattern of antisocial behavior (Frick et al., 2014).
The inappropriate behaviors discussed in the foregoing section are also called disruptive behaviors. These behaviors not only disrupt child–child interactions and child–adult interactions, but when these behaviors occur frequently, the relations among children and the relations between children and adults are disrupted as well. “Externalizing behavior” is another general term for these inappropriate behaviors (Achenbach & Edelbrock, 1978). It is used to distinguish these behaviors from overcontrolled or internalizing behaviors such as withdrawal and anxious behaviors.
In addition to the ones discussed in the preceding section, there are more types of behaviors that are disruptive. Impulsive behaviors such as interrupting others and having difficulty in waiting turn indeed are clearly disruptive. Hyperactive behavior, such as running about in the living room or leaving one’s seat in the classroom and during meals at home, are troublesome as well. Finally, attention problems may occur unnoticed such as the difficulty in sustaining attention, but other behaviors related to attention problems such as not following through on instructions are quite upsetting.
Disruptive behaviors may occur either infrequently or in isolation in individual children and in these cases the behaviors can then be considered as “normative.” However, they may also occur as clusters. These clusters of co‐occurring patterns of inappropriate behaviors or syndromes form the basis of the psychiatric categories from the classification systems DSM‐5 (American Psychiatric Association, 2013) and ICD‐10 (World Health Organization, 1996). Although these syndromes originate from hypotheses about covarying symptoms or behaviors derived from observations of patients by clinicians, factor analytic studies of child and adolescent problem behavior support how these behaviors are associated to each other (see further in this chapter).
In the DSM‐IV, the two Disruptive Behavior Disorders (DBDs), i.e., oppositional defiant disorder (ODD) and conduct disorder (CD), with characteristic features of oppositional, aggressive, and antisocial behavior, were included in the chapter titled “Attention Deficit and Disruptive Behavior Disorders.” Attention deficit/hyperactivity disorder (ADHD), with characteristic features of hyperactive behavior, impulsive behavior, and attention problems, also was included in this chapter. In the DSM‐5, however, ADHD is included in the chapter titled “Neurodevelopmental Disorder” while ODD and CD are part of the disruptive, impulse‐control, and conduct disorders together with pyromania, kleptomania, and intermittent explosive disorder.
Although the subject of this book is ODD and CD, we also will pay attention to ADHD. ODD and CD are related to ADHD with respect to their symptoms, and these disorders also often co‐occur or are comorbid. Indeed, the odds ratio of ODD/CD–ADHD comorbidity in a meta‐analysis of community‐based samples is 10.7 (Angold, Costello, & Erkanli, 1999). Or, to put it in another way, about 50% of children and adolescents with ODD or CD are comorbid with ADHD, and vice versa (Kutscher et al., 2004). Therefore, in the assessment of children who are referred because of disruptive behavior problems, clinicians need to consider whether the child’s inappropriate behaviors or symptoms are part of ODD or CD, of ADHD, or whether both disorders can be diagnosed. As we will describe in later chapters, the treatment of ODD or CD comorbid with ADHD is different from the treatment of ODD or CD without ADHD comorbidity.
In DSM‐5, ODD is defined as a pattern of angry/irritable mood, argumentative/defiant behavior, or vindictiveness lasting at least 6 months as evidenced by at least four symptoms. These symptoms are exhibited during interaction with at least one individual who is not a sibling. The persistence and frequency of the symptoms should exceed what is normative for the individual’s age, gender, and culture. For children younger than 5 years, the behavior should occur on most days whereas for children aged 5 years and above, the behavior should occur at least once a week. However, for the vindictiveness symptom, it is specified that the behavior must occur at least twice in the past 6 months.
DSM‐5 differentiates the various symptoms of ODD in three distinct groups (see Box 1.1). Research has shown that symptoms of negative mood and affective dysregulation (angry/irritable mood) can be distinguished from symptoms of headstrong, antagonistic, and oppositional behavior (argumentative/defiant behavior) (Rowe, Costello, Angold, Copeland, & Maughan, 2010), while studies disagree as to whether being spiteful (vindictiveness) loads on the behavioral dimension or not (Rowe et al., 2010; Stringaris & Goodman, 2009a). According to a recent study in young children, ODD is best characterized as two separate dimensions, one behavioral and one affective, which are comparable for both boys and girls (Lavigne, Bryant, Hoplins, & Gouze, 2015). Similarly, results from a study in elementary school children support a two‐factor model composed of oppositionality and irritability; findings were robust across gender (Herzhoff & Tackett, in press). Thus, based on the heterogeneity of symptoms, ODD may be considered a mixed disorder of behavior and emotion (Matthys, Vanderschuren, Schutter, & Lochman, 2012; Matthys, Vanderschuren, & Schutter, 2013), a characteristic to which both clinicians and researchers need to pay more attention.
Is touchy or easily annoyed
Is angry and resentful
Argues with adults
Actively defies or refuses to comply with adult’s requests or rules
Deliberately annoys others
Blames others for his or her mistake or misbehavior
Is spiteful or vindictive
According to DSM‐5, symptoms need not be present in more than one setting (e.g., either at home, at school, or in the community). However, if symptoms are confined to only one setting (e.g., only at home) the disorder is considered mild. If some symptoms are present in at least two settings (e.g., both at home and at school), the severity of the disorder is specified as moderate, and if some symptoms are present in three or more settings, the disorder is specified as severe (for a case, see Box 1.2).
The prevalence of ODD, that is, the percentage of cases at a given point in time, varies considerably between studies with a median of 3.2 (Lahey, Miller, Gordon, & Riley, 1999). Although gender differences for ODD are quite inconsistent across studies, most data suggest either somewhat higher rates in boys than in girls or no gender difference (Loeber, Burke, Lahey, Winters, & Zera, 2000).
Jimmy, an 8‐year‐old boy, was brought to the clinic by his single mother. She reported that he is often disobedient toward her at home and exhibits tantrums. Because of this misbehavior, she has become exhausted. According to his mother, Jimmy has been problematic from early childhood on. As an infant he often cried, and as a toddler he was irritable, highly distractible, and hyperactive. From age 3 years on, Jimmy also began to exhibit temper tantrums. Presently, these outbursts occur twice a week, usually in the context of being asked to complete chores. His refusal can escalate from ignoring to shouting, and crying. In addition, he argues with mother on an almost daily basis. At school, Jimmy has difficulty following the teacher’s directions; these need to be repeated multiple times. He is easily provoked by peers and may deliberately annoy them. As a result, he has developed a poor reputation among his classmates. He gets good grades, and his distractibility and hyperactivity have decreased over the years.
In DSM‐5, CD is characterized as a repetitive and persistent pattern of behavior in which the basic rights of others or major age‐appropriate societal norms or rules are violated as evidenced by at least three symptoms in the past 12 months with at least one symptom in the past 6 months. Four groups of symptoms are distinguished: (1) aggression to people and animals, (2) destruction of property, (3) deceitfulness or theft, and (4) serious violations of rules (see Box 1.3). Some of the symptoms of CD, however, are not within the capacity of preschool children to perform such as forcible sexual activity, use of weapons, and breaking into houses (Wakschlag et al., 2007).
Aggression to people and animals
Bullies, threatens, or intimidates others
Initiates physical fights
Is physically cruel to people
Is physically cruel to animals
Steals while confronting a victim
Forces someone into sexual activity
Destruction of property
Destroys others’ property
Deceitfulness or theft
Breaks into someone else’s house, building, or car
Lies to obtain goods or favors to avoid obligations
Steals without confronting a victim
Serious violations of rules
Stays out at night
Runs away from home
Truant from school
DSM‐5 distinguishes three types of CD: the childhood‐onset type with onset of at least one symptom prior to age 10 years, the adolescent‐onset type with absence of any symptom prior to age 10 years, and the unspecified onset type; in the latter, there is not enough information available to determine whether the onset of the first symptom was before or after age 10 years. The distinction between the childhood‐onset and the adolescent‐onset type is supported by various studies (Moffitt, 1993; Lahey et al., 1998; Silberg, Moore, & Rutter, 2015; for a review, see Moffitt, 2003; see also Chapter 2). Childhood‐onset CD, but not adolescent‐onset CD, is significantly associated with ADHD (Silberg et al., 2015). Childhood‐onset CD has also been shown to have relatively poorer outcomes when compared with adolescence‐onset CD. Because of this outcome, the childhood‐onset type has also been called the life‐persistent CD subtype (Moffitt, 2003).
Some longitudinal studies suggest that among the children with early‐onset CD some lack the continuity of conduct problems from childhood to adulthood; these children therefore have been termed “childhood limited conduct problem group” (Moffitt, 2003). However, to date there is not enough evidence to further divide the childhood‐onset type into a life‐course‐persistent versus a childhood‐limited group (Moffitt et al., 2008).
Compared with the DSM‐IV, in DSM‐5 the specifier “limited prosocial emotions” is added. To qualify for this specifier, the child must display at least two of the four characteristics persistently over at least 12 months and in multiple relationships and settings: lack of remorse or guilt, callous lack of empathy, unconcerned about performance, and shallow or deficient affect. These characteristics have been extensively studied in older children and adolescents as callous–unemotional traits (for a review, see Frick et al., 2014). There is emergent evidence that low concern and callousness are evident at preschool age and have predictive utility for conduct problems (Ezpeleta, de la Osa, Granero, Penelo, & Domenech, 2013) (for a case study, see Box 1.4).
Jeffrey, a 13‐year‐old boy, was referred by his family doctor to the community mental health center for concerns regarding his aggressive behavior toward his younger brother and sister. According to his mother and stepfather, Jeffrey has always been a stubborn child who wanted to get his own way, but his behavior problems have escalated over the past 2 years. He began hurting his younger brother intentionally, tyrannizing his younger sister, and hiding her personal belongings. Jeffrey also has been bullying a smaller classmate at school. This child reported to his teacher that Jeffrey threatened to beat him. Over the past year, Jeffrey has had several arguments with teachers. His school performance has declined but this does not seem to touch him. Both his parents and teachers are concerned that Jeffrey seems to enjoy hurting others and does not demonstrate any remorse for his actions. Some weeks ago, Jeffrey was caught stealing money from his mother’s wallet. His stepfather also suspects him of lying.
ODD and CD are both related to, and are different from, each other. The association among these disorders needs to be considered from a developmental point of view. Longitudinal studies not only give the opportunity to prospectively follow children (follow‐forward studies) but also to examine earlier diagnoses in adolescents who meet criteria of disorders (follow‐back studies). With regard to the relation between ODD and CD, in the study by Rowe and colleagues (2010), it was shown that most children with ODD did not develop CD, whereas only a small group of CD children had prior ODD. Regarding the further development of these disorders, elementary school children with ODD are not only at risk for CD in early adulthood but also for anxiety disorders and depression in adolescence and (early) adulthood (Rowe et al., 2010; Stringaris & Goodman, 2009b). Based on the results of studies on the dimensions of ODD there is evidence that the association between ODD and depression or anxiety may be explained by the shared negative affectivity and the irritability component, whereas the association between ODD and CD may be explained by the shared headstrong and oppositionality component (Burke et al., 2010; Rowe et al., 2010; Stringaris & Goodman, 2009b). In conclusion, ODD and CD seem to represent related though distinct forms of psychopathology. In this context, it is import to mention that DSM‐5 allows for concurrent comorbidity among ODD and CD.
The prevalence of CD varies considerably between studies with a median of 2.0; CD is more common in boys than in girls (Lahey et al., 1999).
Other diagnostic categories are relevant here for two reasons. First, some characteristic behaviors of these disorders are similar to ODD and CD symptoms. For example, “Does not seem to listen when spoken to directly” is a criterion of inattention in ADHD but is related to refusing to comply with adults’ requests, which is a symptom of ODD. Also, a depressed mood, characteristic of a dysthymic disorder, in children may manifest as irritability in children. This expression of irritability may be related to the ODD symptom “Is often angry and resentful.” Other related disorders are disruptive mood dysregulation disorder and intermittent explosive disorder. Differential diagnosis of these disorders and ODD or CD will be discussed in the chapter on assessment (Chapter 6).
Second, some disorders often co‐occur with ODD and CD. The most prevalent comorbid disorder of ODD and CD is ADHD (Angold et al., 1999). This comorbidity is highly important both with respect to the etiology of ODD and CD (Chapters 3 and 4), the assessment (Chapter 6), and the treatment of ODD and CD (Chapters 8, 9, 10, and 11) (for a case, see Box 1.5). DSM‐5 provides lists for two types of ADHD symptoms: inattention and hyperactivity/impulsivity. Thus, three ADHD subtypes are distinguished: predominantly inattentive presentation (I), predominantly hyperactive/impulsive presentation (HI), and combined presentation (C).
Tim, a 6‐year‐old boy, was referred by his pediatrician to the clinic. His mother and father report that Jim has always been a willful child, but difficulties have worsened over the past year. As a toddler, he was in constant motion. Even up to the age of 5 years, he got into dangerous situations such as suddenly running out of the pavement into the road without looking. Nevertheless, what is currently the most stressful for his parents is Tim’s argumentative and defiant behavior. Every request turns into a battle where he can be angry, and sometimes this leads to a tantrum. It is also very tiring that he does not stop talking. At school, Tim often leaves his place in the classroom. If he does not get his way with other children, he gets angry. Tim is also easily annoyed by peers.
Comorbidity with other disorders is important as well, including mood disorders (Angold et al., 1999) such as dysthymic disorder, anxiety disorders (Angold et al., 1999) such as separation anxiety disorder, specific learning disorder, and academic underachievement (Frick et al., 1991; Hinshaw, Lahey, & Hart, 1993), and communication disorders. We will pay attention to these comorbid disorders when discussing the assessment (Chapter 6) and treatment of ODD and CD (Chapters 8, 9, 10, and 11).
Besides DSM‐5, the International Classification of Diseases ICD‐10 (1996) of the World Health Organization is used in many countries. It should be noted, however, that little research has been conducted into ICD‐10 defined disruptive behavior problems. Over the years, differences between both systems have diminished, but important distinctions still remain. Currently, ICD‐11 is being developed. Here, we present some recommendations regarding the disruptive behavior and dissocial disorders by a task group (Lochman et al., 2015). First, while ODD is not a distinct category in ICD‐10, but is instead a subtype of CD, ODD is a distinct category in ICD‐11 along with conduct dissocial disorder. Second, disruptive mood dysregulation disorder, with severe irritability as the core feature, is not a diagnostic category because of limited reliability, a lack of psychiatric consensus, and very high rates of overlap with other disorders. Instead, the task group proposed that ICD‐11 include a specifier to indicate whether or not the presentation of ODD includes chronic irritability and anger. Third, the qualifier limited prosocial emotions is applied not only to conduct dissocial disorder but to oppositional defiant disorder as well.
Factor analytic studies support the distinction DSM makes between ODD, CD, ADHD, and other disorders. Factor analysis is one statistical technique that can search for patterns in covariation among a group of behaviors. Using exploratory and confirmatory factor analyses with data from various questionnaires in population and clinically referred samples, Hartman et al. (2001) investigated the internal construct validity of a DSM‐IV based model of ADHD (inattention, hyperactivity/impulsivity), ODD, CD, generalized anxiety, and depression. The factorial structure of these syndromes was supported by the data. However, the DSM‐IV model did not meet the absolute standard of adequate model fit, leaving substantial room for improvement. Findings from another study support the four‐factor DSM‐IV model (ODD, CD, hyperactivity/impulsivity, inattention) of the DSM‐IV‐TR “Attention Deficit and Disruptive Behavior Disorders” (Lahey et al., 2008). In addition, this model fitted better with the data than models based on ICD‐10 and the Child Behavior Checklist (Achenbach, 1991).
It is also important to pay attention to appropriate behaviors, because children with ODD and CD may not have these behaviors in their repertoire. There are a number of social behavior skills that children use to cope adequately with everyday problem situations. These skills include entering a group, starting a conversation, asking questions, and listening to others. Other socially appropriate behaviors such as showing interest, comforting, sharing, helping, and donating are more clearly prosocial in that they are intended to benefit other persons (Eisenberg & Fabes, 1998). Empathy is related to prosocial behavior (Eisenberg & Miller, 1987). Empathy is defined as the understanding of, and sharing in, another’s emotional state (Hoffmann, 2000). Strictly speaking, empathy is not behavior. Instead, it is an emotion. Empathy involves a matching of emotions between the child and the other person, that is, feeling with another person.
Among oppositional behaviors, one may distinguish passive forms of noncompliance (e.g., ignoring parental commands) from mild active forms (e.g., refusing parental directions) and severe active forms or defiant behavior (e.g., angrily reject parental commands).
Aggressive behavior may manifest in various forms: in physical aggression, in verbal aggression, and in relational aggression, that is, damaging interpersonal relationships or feelings of inclusion.
When considering the underlying motivation in aggressive behavior, the distinction can be made between reactive, defensive, or “hot‐blooded” aggression and proactive, controlled, or “cold‐blooded” aggression.
The affective factor of psychopathy, callous–unemotional traits, consists of a lack of guilt and remorse, a lack of concerns for the feelings of others, shallow or superficial expressions of emotions, and a lack of concern regarding performance in important activities. These characteristics are included in DSM‐5 as limited prosocial emotions, a specifier of CD.
In ODD, symptoms of negative mood and affective dysregulation (angry/irritable mood) can be distinguished from symptoms of headstrong, antagonistic, and oppositional behavior (argumentative/defiant behavior).
In CD, the distinction is made between the childhood‐onset type of CD, with onset of at least one symptom prior to age 10 years, the adolescent‐onset type of CD, with absence of any symptom prior to age 10 years, and the unspecified onset type.
Some symptoms of ADHD are clearly disruptive such as not following through on instructions (inattention), leaving seat in the classroom (hyperactive behavior), and difficulty waiting turn (impulsive behavior).
The most prevalent comorbid disorder of ODD and CD is ADHD.
This chapter will describe the normal development of child compliance, anger, and aggression during the childhood years. Using a developmental psychopathology perspective, the developmental course of the problematic externalizing behaviors that comprise ODD and CD, and of subtypes of children with conduct problems will be described. A specific focus will be placed on the development of early starters versus late starters, of aggressive–rejected versus aggressive–nonrejected children, of children with psychopathic traits, and will emphasize the ongoing development of the disordered behavioral patterns into young adulthood (Conduct Problems, Prevention Research Group, 2014).
The first textbook on developmental psychopathology appeared in 1974 (Achenbach, 1974). In the subsequent 35 years, the benefits of using a developmental psychopathology approach have been increasingly recognized by the scientific community, and research on the causative factors and treatment of childhood mental health difficulties such as conduct problems has been advanced and expanded under a developmental psychopathology framework (Lochman, Boxmeyer, & Powell, 2008). Developmental psychopathology includes a focus on understanding causal processes, understanding the role of development, and understanding the continuities and discontinuities between normality and pathology.
The need for a developmental psychopathology approach arose out of growing recognition of the limitations of traditional developmental and psychological perspectives (Rutter & Sroufe, 2000). At the same time, emerging research demonstrated that developmental pathways were complex and dynamic, involving both individual and environmental factors (Rutter & Sroufe, 2000). An inclusive framework allowing examination of a multitude of factors on developmental processes was needed to integrate the growing bodies of information from diverse disciplines such as psychology, psychiatry, sociology, biology, and epidemiology. By taking a developmental psychopathology perspective, clinicians and researchers attempt to understand the processes by which behaviors, both normative and atypical, arise and are maintained (Luthar, Burack, Cicchetti, & Weisz, 1997). Developmental psychopathology research in the past 25 years has begun to move beyond knowing the correlates of disorders to increasing understanding of the pathways leading to specific disorders such as ODD and CD. In recent years, we have learned much about how the brain and neurobiological processes develop in the context of children’s experiences in their world (Doom & Gunnar, 2013; Sroufe, 2013), and how these interactions between child factors and context factors contribute to the later antisocial behavior of adolescents and adults (Hyde, Shaw, & Hariri, 2013).
The recent focus on Research Domain Criteria (RDoc) from the National Institute of mental Health has attempted to target underlying processes which may be part of the etiology of various types of psychopathology (Lenzenweger, 2013). These underlying processes, occurring before the full phenotype of the disorder can be observed, have been considered to be endophenotypes and intermediate phenotypes (Lenzenweger, 2013). These underlying processes are complex. Research to date has not found localized, dedicated neural circuits for specific emotions such as fear; instead the presence of an emotion such as fear may be best understood by studying the ingredients of the emotional response (Franklin, Jamieson, Glenn, & Nock, 2015). Thus, the development of externalizing problems in children can be seen to develop from high levels of impulsivity in early childhood due to many genetic loci and their interactions with each other and the neonatal early environment, which elicits complex bidirectional transactions between children and their environment over time (for example, eliciting hostile and inconsistent parenting in the childhood and adolescent years). Families can unwittingly provide operant reinforcement for children’s dysregulated behavior. In the presence of compromised prefrontal brain functioning, “top‐down” inhibitory control does not occur, and the individual’s emotional and behavioral problems escalate over time (Beauchaine & McNulty, 2013), into young adulthood and beyond.
Knowledge of normative developmental processes and behaviors corresponding to different developmental levels is imperative to understanding maladaptive behaviors and disorders (Essau & Petermann, 1997; Sroufe & Rutter, 1984). At the most basic level, problems can be identified by making contrasts between expected abilities at a given stage of development and an individual’s actual functioning (Edelbrock, 1984). For example, physical aggression is common among preschoolers, but steadily declines with age to become almost nonexistent by the end of adolescence (Bongers, Koot, Van der Ende, & Verhulst, 2004). In a developmental psychopathology perspective, the processes involved in development are also crucial considerations. For example, the influences of genetic and environmental factors, such as those emanating from parents (Calkins, Propper, & Mills‐Koonce, 2013), must be analyzed in the context of ongoing developmental processes within the individual (Rutter & Sroufe, 2000). There are normal variations at points in time, and especially over time, in externalizing behaviors and related emotional characteristics (Lochman, Boxmeyer, et al., 2008). In this section, we will review normal development of compliance behaviors, anger, and aggression in children, all of which are key characteristics associated with the DBDs.
All children at every age exhibit both compliant and noncompliant behaviors, and noncompliant behaviors, by themselves, are not causes for referral or diagnosis (Lochman, Barry, Powell, Boxmeyer, & Holmes, 2008). However, the frequency, intensity, form, and effects of such behaviors set apart normal and abnormal expressions of noncompliance. Clinical manifestations of noncompliance are categorized as symptoms of ODD and were described in the preceding chapter. Child noncompliance is the main reason why parents seek help from mental health providers (Chamberlain & Smith, 2003). This section will outline patterns and contributing factors in normal compliant and noncompliant behaviors.
Though most people involved in the care of children have an idea of what is meant by compliance and noncompliance, these behaviors often prove difficult to define operationally. In their treatment manual for noncompliant children, McMahon and Forehand (2003) use the definition “appropriate following of an instruction within a reasonable and/or designated time” to operationalize compliance, noting that it is important to distinguish between the initiation of compliance and the completion of the specified task (Schoen, 1983). A range of 5–15 seconds is suggested as a reasonable period for the initiation of compliance. McMahon and Forehand (2003) define noncompliance as the refusal to initiate or complete a request and/or the failure to follow a previously stated rule that is currently in effect.
Children’s compliance behaviors are affected by their family context, as maternal guidance and discipline behaviors are clearly related to children’s compliance (Kok et al., 2013; Kwon & Elicker, 2012), although other mediation and moderation factors are often involved. For example, greater family stress during pregnancy has been found to predict reduced child compliance, but that effect was mediated by family stress’s effect on reducing maternal positive discipline (Kok et al., 2013). Children’s characteristics further complicate these relationships, as children with COMT Met alleles are more susceptible to the protective effects of maternal positive discipline; children with these alleles have greater activation in the orbitofrontal cortex when receiving a reward, in comparison to other children (Kok et al., 2013). However, use of praise alone does not appear to be very effective, as a meta‐analysis of 41 studies examining compliance in children ranging in age from 18 months to 11 years, found that reprimands and nonverbal responses, and not praise alone, contributed to consistently greater compliance (Owen, Slep, & Heyman, 2012).
Other child characteristics that affect compliance have been evident in research that has shown that toddlers who are more securely attached to their mothers and who were low in temperamental negative reactivity display better compliance (Lickenbrock et al., 2013). Similarly, Kochanska and Kim (2013) have found that when children have difficult temperaments (poor effortful control and high anger proneness), unresponsive and inconsistent parenting is more likely to lead to child noncompliance. In defining compliance and noncompliance, it is also important to recognize that these are not stand‐alone behaviors by the child, but are bidirectional interactional processes between adult and child. Parenting behaviors can affect a child’s likelihood of compliance, and child characteristics and responses can in turn affect parenting behaviors.
Children first begin to understand the consequences of their own behavior between 6 and 9 months of age, and may also learn to recognize the word “no” during this time. Increasing physical development, cognitive abilities, social skills, and receptive language skills lead to improved abilities to respond to verbal directions, and children are generally able to follow simple instructions by age two. Nonetheless, noncompliance with commands is very common for 2‐ and 3‐year‐old children, possibly due to parental expectations (i.e., “the terrible twos”) and parents’ failure to train their young children to comply (Brumfield & Roberts, 1998).
Compliance levels are expected to increase with age in typically developing children (e.g., Lahey et al., 2000). Although specific normative data for compliance does not exist because of complex issues about sample characteristics and measurement (Brumfield & Roberts, 1998; McMahon & Forehand, 2003), the expected progression of compliant behaviors in young children as they age has generally been found, with some exceptions (Smith, Calkins, Keane, Anastopoulos, & Shelton, 2004). Vaughn, Kopp, and Krakow (1984) reported increases in compliance with maternal requests between 18 and 30 months of age and Kochanska, Coy, and Murray (2001) found that children’s committed, eager compliance to maternal directions increased from 27% to 56% during the 14–33 months of age period. Brumfield and Roberts (1998) reported that, whereas 2‐ and 3‐year‐old children only complied with 32.2% of maternal commands, the compliance rate for 4‐ and 5‐year‐olds reached 77.7%. However, Kuczynski and Kochanska (1990) reported no change in overall compliance to maternal requests between toddlers (1.5–3.5 years old) and 5‐year‐olds, but did find variations over time in specific noncompliant behaviors. Direct defiance and passive noncompliance decreased with age, and simple refusal and negotiation (an indirect form of noncompliance that required more autonomy and social skill) increased.
By the time they reach school age, children are expected to comply with adult requests the majority of the time. McMahon and Forehand’s (2003) review of studies suggests that compliance rates are around 80% for normally developing children. Patterson and Forgatch (1987), however, report lower compliance rates in a sample of “nonproblem 10‐ and 11‐year‐old boys”: 57% to maternal requests and 47% to paternal requests.
In adolescence, noncompliant behaviors often again increase, and go above childhood levels in typically developing youth. Developmental changes in cognition and social skills, combined with adolescents’ growing independence and need to establish their own identity, may lead to increased parent–adolescent conflict. Conflict tends to be at its highest during early adolescence, and to decline from early adolescence to mid‐adolescence and from mid‐adolescence to late adolescence (Laursen, Coy, & Collins, 1998).
Boys and girls differ in their normative rates of oppositional, noncompliant behaviors, with boys demonstrating higher levels than girls do during childhood. However, the gender difference closes with age, and boys and girls demonstrate increasingly similar rates, as they progress through adolescence (Bongers et al., 2004).
Like compliance and noncompliance, angry and aggressive behaviors are common to all children, representing clinically significant problems only when frequent and severe enough to disrupt a child or family’s daily life (Lochman, Barry, et al., 2008; Lochman, Barry, Powell, & Young, 2010). Not surprisingly, children who are identified as angry by parents and teachers are more likely to display externalizing behaviors (e.g., Bohnert, Crnic, & Lim, 2003; Denham et al., 2002; Rydell, Berlin, & Bohlin, 2003). Relations have also been reported between child anger and internalizing problems (e.g., Eisenberg et al., 2005) and between child anger and being victimized by peers (Hanish et al., 2004).
The presence or absence of anger is often a defining factor in the classification of aggression. Anger is a key feature of hostile aggression, which carries the intent to harm and is accompanied by emotional arousal, but not instrumental aggression, which is motivated by external reward rather than by emotional arousal. In a similar distinction, as noted in Chapter 1, reactive aggression is emotionally driven and takes the form of angry temper outbursts (Hubbard, McAuliffe, Morrow, & Romano, 2010), whereas proactive aggression is instrumentally driven and takes the form of goal‐driven behaviors (e.g., domination of others or obtaining a desired object; Dodge, 1991; Dodge, Lochman, Harnish, Bates, & Pettit, 1997).
Anger and development
Tysiące ebooków i audiobooków
Ich liczba ciągle rośnie, a Ty masz gwarancję niezmiennej ceny.
Napisali o nas:
Nowy sposób na e-księgarnię
Czytelnicy nie wierzą
Legimi idzie na całość
Projekt Legimi wielkim wydarzeniem
Spotify for ebooks