Despite the fact that terms such as “evidence-based practice”, “empirically supported treatments”, and “best practices” have become ubiquitous in education as well as other human services disciplines their definitions are not generally agreed upon, and worse, the principles are frequently not applied (1). Whatever the name that is used, the requirement for scientific knowledge to function as a filter for recommendations regarding treatment should be a fundamental principle guiding clinical work. Science has become the dominant judge of value in many disciplines, but in psychology interventions based on scientific evidence are often not sought by consumers, and – even worse – not endorsed by service providers. Dubious interventions and pseudo treatments are especially popular in the area of developmental disabilities (2). In the following paragraphs, we will 1) describe the scientific and pseudoscientific approach, 2) list the reasons for the proliferation of unsubstantiated claims in autism treatment, 3) summarize the results of the latest report on educational interventions for children with autism prepared by the National Autism Center, and 4) offer guidelines for clinicians regarding their own practice treating autism.

Science is virtually the only means by which interested individuals can test and evaluate the effectiveness or any proposed intervention (3). The premises of the scientific method are few and straightforward. They include (but are not limited to) 1) conducting quantitative and direct measurements of observable events, 2) testing of verifiable hypotheses, 3) using analytic means to establish relations between events, 4) relying on data to draw conclusions, 5) insisting on replication, 6) publishing of results in peer reviewed journals, and 7) self-correcting their work based on the above (4). Through the scientific process, assertions are confirmed or disconfirmed resulting in an accumulated body of knowledge and technology. Our understanding of phenomena under study is constantly changing as new facts come in from additional research. Despite the fact, that the scientific method has brought many contributions to society, it is often distrusted and disdained (5). The reasons are related to the fact that science does not offer “quick fixes” – the accumulation of evidence is an arduous and slow process. The methods of conducting experiments and analyzing results are not well understood by lay people. Experiments sometimes do not conform to conventional wisdom and common sense and at times seem to defy logic. Moreover, since scientific inquiry is painstaking, gathering data piece-by-piece may appear to be a self-serving endeavor, detached from real-world problems. Finally, science does not offer “good” or “bad” information with unequivocal conclusions – facts are facts, they should have neutral value. This is difficult to embrace by audiences who are hoping for a much-wanted answer, simple, and unambiguous. These characteristics of science are apparent in the scientific findings on climate change, where despite an accumulation of evidence based on the preponderance of careful scientific research, politicians and other non-scientists choose to ignore the findings.

In contrast to science, pseudoscience has many compelling features. They include 1) portraying their interventions as extraordinarily effective with little or no risk associated with their use, 2) basing claims on anecdotes, case studies, surveys, and other non-experimental methods, 3) using methods of inquiry that include uncontrolled, descriptive studies with qualitative data, 4) disseminating results directly to consumers via Internet, mass-media, brochures, reports in self-published books and journals and without peer-review, 5) promoting interventions through endorsements by individuals with apparent credibility or by celebrities (4, 6). Pseudoscience is, unfortunately, very wide-spread in the area of human services. It is also a big business for providers of the pseudoscience. Because of their often desperate need for help for their children, the population of parents of children with autism is especially vulnerable to interventions touted as “cures” or “breakthroughs” (7). Because it has attracted so many treatments of dubious merit, autism is called “a late 20^th century fad magnet” by Metz, Mulick and Butter (8). What may be some reasons for such desperation and gullibility of parents of autistic children?

First, the nature of autism makes it a fertile ground for pseudoscience. Despite the fact that this disability was described 70 years ago by Kanner (9), it is still rather unknown and perplexing (10). Autism is treatable, but incurable (11). Thus, parents who are told that their child has a life-long disability, that is quite severe and puzzling reach out for everything and anything that may help. Additionally, parents of autistic children experience much more stress than parents of children with other disabilities (12) and this disorder often has far-reaching adverse effects on the whole family, which motivates parents to find anything that will remedy this situation. Further, since autism is not distinguished by any physical abnormalities, children with this disorder do not look disabled. They also act typical in many instances and show, in some areas, exceptional abilities (13). It is easy, thus, to believe that the child is “almost normal” and will reveal himself or herself with some sort of special individual or technique. Maurice (14) points out the myth of “hidden inner child.” That is the myth that there is a normal child hidden under the autism who may be awakened if only the right conditions happen. This and other misconceptions of autism encourage parents to undertake treatments that offer quick cures. Heterogeneity of the population of children with autism is also one reason for popularity of nonscientific therapies. Because of the inexact and ever-evolving diagnostic criteria for autism, children with autism exhibit a variety of different symptoms and, as individuals, they also are likely to be subjected to at least several therapies (15), in a way as to make it impossible to determine what the cause of the change is. If there is a testimonial supporting one therapy stating that child X advanced, parent of child Y may also want to subject his child to the same treatment that child X received, even though it is not known whether this treatment had anything to do with the changes in that one child. There are also additional reasons, not particular to autism, for the popularity of nonscientific therapies but they certainly also affect the field of developmental disabilities (16). They include the following: the incomplete effectiveness of available therapies, the best available treatment is onerous for clients, alternative treatments are supported by ideology, the treatment is promoted by proprietary professional group, and the treatment is recommended by authority or chosen on the grounds of tenacity.

A good measure of the success parents have in their child’s improvement is choosing an appropriate treatment. However, how to choose an effective evidence-based practice is not easy for most parents. It is not feasible to expect most of the parents to be able to evaluate the strength of research evidence on their own. A much better way is to inform consumers about the current status of practice guidelines for autism intervention. Such guidelines are established according to recommendations of teams of experts based on criteria for “empirically supported therapies” (EST). In 1995, a Division 12 (Clinical Psychology) Task Force on Promotion and Dissemination of Psychological Procedures and the American Psychological Association (APA) Task Force on Psychological Intervention Guidelines created three categories to describe degrees of evidence: Well-established Treatments, Probably Efficacious Treatments, and Experimental Treatments to describe the degree of evidence we have on the effectiveness of treatments. The information below comes from Chambless and Ollendick (17).

At least two good between-group design experiments must demonstrate efficacy in one or more of the following ways:

– Superiority to pill or psychotherapy placebo or to other treatments,

– Equivalence to already established treatment with adequate sample sizes.

OR

A large series of single-case design experiments must demonstrate efficacy with both of the following:

– Use of good experimental design,

– Comparison of intervention to another treatment.

AND

– Experiments must be conducted with treatment manuals or equivalent clear description of treatment,

– Characteristics of samples must be specified,

– Effects must be demonstrated by at least two different investigators or teams,

Two experiments must show that the treatment is superior to a waiting-list control group.

OR

One or more experiments must meet criteria Ia or Ib, III, and IV for well-established treatments (Category I), but the criterion 5 is not met.

OR

A small series of single-case design experiments must meet criteria II, III, and IV for well-established treatments.

Treatment not yet tested in trials meeting task force methodology criteria.

Using the above-mentioned hierarchy of evidence method to make recommendations for interventions with strong scientific support must be part of professional practice guidelines. The ultimate goals is to set higher standards for care and to make better use of research findings to benefit consumers. The methodology to develop practice guidelines has been applied to autism only since the late 1990s. The first three examples of reviewing a variety of treatments for children with autism were: the New York State Department of Health Early Intervention Clinical Practice Guidelines (18), the Report of the Maine Administrators of Services for Children with Disabilities Autism Task Force (19), and the Recommendations of the Committee on Educational Interventions for Children with Autism of the National Research Council (20). Although these reports represented a very important step forward for evidence-based practice in autism treatment, their utility is limited due to 1) insufficient transparency of the process of guidelines’ creation, 2) incomplete comprehensiveness of reviewed studies, and 3) at this point in time – simply being out-of-date (7). Thus, we will describe in detail the latest effort to establish evidence-based practice guidelines dedicated to parents of individuals with autism and professionals working in the field of developmental disabilities – the National Standards Project undertaken by the National Autism Center. The goals of the project were four-fold: 1) to provide the strength of evidence supporting educational and behavioral treatments that target the core characteristics of autism spectrum disorders (ASD), 2) to describe the age, diagnosis, and skills/behaviors targeted for improvement associated with treatment options, 3) to identify the limitations of the current body of research on autism treatment, and 4) to offer recommendations for engaging in evidence-based practice for ASD (21). The report addressed some of the limitations of previous publications by reviewing the educational and behavioral treatment literature that targeted the core characteristics and associated symptoms of ASD and was published between 1957 and the 2007. Additionally, the report provided information about treatment effectiveness based on age, diagnostic groups, and treatment targets. Finally, the process of creating practice guidelines was completely transparent.

The report included 775 research studies which were reviewed by a group of experts. The inclusion criteria for studies were as follows: 1) the studies involved individuals with ASD (Autistic Disorder, Asperger’s Syndrome, and Pervasive Developmental Disorder – Not Otherwise Specified; PDD-NOS) under age 22; 2) they were published in peer reviewed journals prior to Fall 2007; 3) treatments could be implemented in or by school systems, or early intervention, home-, hospital-, and community-based programs; 4) interventions were educational and behavioral treatments – biomedical interventions (i.e., medication trials, nutritional supplement studies, and complementary and alternative medical interventions) were largely excluded, with the exception of curative diets; 5) studies needed to have empirical data; 6) research was published in English.

The Scientific Merit Rating Scale (SMRS) was developed as a means of objectively evaluating if the methods used in each study were strong enough to determine whether or not a treatment was effective for participants on the autism spectrum. The SMRS involved five critical dimensions of experimental rigor: 1) research design (i.e., degree to which experimental control was demonstrated), 2) measurement of the dependent variable (i.e., the extent to which accurate and reliable data were collected and these data represent the most direct and comprehensive sample of the target skill or behavior that is possible) 3) measurement of the independent variable (i.e., the extent to which treatment fidelity was adequately established), 4) participant ascertainment (i.e., the degree to which well-established diagnostic tools and procedures were used to determine eligibility for participant inclusion in the study), and 5) generalization (i.e., the extent to which researchers attempted to objectively demonstrate the spread of treatment effects across time, settings, stimuli, responses, or persons). For each dimension, reviewers, using clearly defined criteria, assigned a score from 0 (poor score) to 5 (strong score). The dimension scores were compiled to create a composite score for each article. Criteria were also developed to determine if intervention had 1) beneficial treatment effects (i.e., sufficient evidence that favorable outcomes resulted from the treatment), 2) unknown treatment effects (i.e., not enough information to confidently determine the treatment effects), 3) ineffective treatment effects (i.e., sufficient evidence that favorable outcomes did not result from the treatment), and 4) adverse treatment effect (i.e., sufficient evidence that the treatment was associated with harmful effects). All treatment studies were classified into one of 38 treatment categories. The treatment categories are shown in table 2.

A four-level Strength of Evidence Classification System criteria was applied based on the quality, quantity, and consistency of research findings for each treatment category. The four levels in this classification system were 1) Established – sufficient evidence that these treatments produce beneficial effects for individuals with ASD; 2) Emerging – one or more studies shows beneficial treatment effects, but additional high quality studies are needed; 3) Unestablished - little or no evidence about treatment effectiveness; and 4) Ineffective/Harmful – sufficient evidence exists to show treatment is ineffective or harmful. Additionally, each study was evaluated taking into consideration 14 treatment targets: 10 goals in the area of skills increased (i.e., academic, communication, higher cognitive functions, interpersonal, learning readiness, motor, personal responsibility, placement, play, self-regulation), and 4 goals in the area of behaviors decreased (i.e., general symptoms, problem behaviors, repetitive patterns of behavior or interests, sensory or emotional regulation).

The National Standards Project identified 11 Established Treatments, 22 Emerging Treatments, 5 Unestablished Treatments, and zero Ineffectual/Harmful Treatments. The treatments categorized by the National Standards Committee are shown in table 1.

The report concluded with several interesting facts about the type of interventions found to be “established”. First, less than 30% of the reviewed interventions were evaluated as “established.” But within those, about 67% were developed exclusively from the behavioral literature, and of the remaining 33% three-quarters represent treatments for which research support comes predominantly from the behavioral literature. Only less than 10% of established treatments came from theory of mind perspective. “This pattern of findings suggests that treatments from the behavioral literature have the strongest research support at this time” (21). Close to 60% of the reviewed interventions were evaluated as “emerging” and a little over 10% as “unestablished”. There were no interventions which were deemed “harmful” based on the reviewed literature.

The concluding remarks of the report state that parents of children with autism and professionals working with those children should seriously consider established treatments when making decisions about treatment choice. The report, it should be noted, cautioned that these treatments will not universally produce favorable outcomes for all individuals with ASD. Further, the authors suggest that children should not be subjected to the emerging treatments at the beginning of therapy. Instead, they may be an option later in therapy or if established treatments are not appropriate or acceptable for the child’s family. Lastly, unestablished treatments should not be used until additional research has been conducted and has shown favorable effects for children with ASD.

The National Standards Report provides parents and practitioners with solid professional judgments of the kinds of treatment that have evidence of efficacy. However, as the report acknowledges, not all treatments will work uniformly well in all cases. Knowing the results of the latest practice guidelines for treatment of autism, what should practitioners do in their daily work? As O’Donohue (22) suggests they should behave scientifically themselves. That means, they should take an experimental approach to all intervention efforts, empirically establishing the efficacy of the treatment they are using and being willing to change the treatment in light of evidence that it is not working. Adhering to scientific methods is of paramount importance. This means educating oneself on science-based interventions and understanding that employing pseudoscientific treatment not only can be of no benefit but also can cause direct (i.e., health deterioration) or indirect (i.e., loss of time and resources) harm. It also means seeking feedback about clinical work from peers, especially those with greater relevant scientific expertise and giving corrective feedback to those who are failing to use sound practice. Finally it means making data-based decisions and recommending to other professionals and parents of children with ASD sources where they can obtain knowledge on empirically validated treatments. In case of autism, some of such sources are the Association for Science in Autism Treatment (www.asatonline.org), the National Autism Center (www.nationalautismcenter.org), the Autism Help Center of the Cambridge Center for Behavioral Studies (www.behavior.org), and the Autism Special Interest Group of the Association for Behavior Analysis International (www.abainternational.org). Most importantly, however, every clinician should be guided in their professional work by a statement by Leonardo Da Vinci: “Those who fall in love with practice without science are like a sailor who enters a ship without a helm or a compass, and who never can be certain whither he is going” (23).