Editorial: What's the most effective intervention?

How should parents and educators decide what instructional methods to use with individual students?

Please imagine the following Q-&-A:

Question: "Dear Automobile Editor, I'm planning to buy a new (and I mean a brand-spanking new) pick-up truck pretty soon. Currently I drive a 1994 Toyota extra-cab with the fabled four-cylinder engine that lasts forever. My current truck has lots of minor issues (my family and friends calls it "The RustBucket" because the pickup bed is falling apart; one of the door windows rattles; the driver's seat is caving in; etc.). It’ll run for a few more years, but what's the best truck I should buy?"

Answer: "'Best' is maybe the most important word here. And the question about 'best' begs many other questions. Examples of those begged questions: Best for safety? Best for your pocket book? Best for speed? Best for driver comfort? Best for hauling lumber | rock | mulch? Best for driving long distances? Best for longest lasting? Best for off-road vacations? Best looking? Best for wowing my friends and neighbors ? Best for making my friend Michael question his allegiance to European sedans?"

From that list of begged questions, I hope that readers get the idea that there are lots of ways to decide what counts as "best." The question requires that one know the criterion for determining "best" and, because the evaluation is a superlative ("good," "better," and "best"), it implies an orderly comparison of alternatives along some criterion.

Now, please compare the truck-buying question with another inquiry:

Q: "Dear editor, I am working with a student who has autism. What's the best intervention I can use?"

Responding to this question requires that one know (a) the criterion for (b) an orderly comparison of interventions. Who knows whether “best” means “most liked,” “most effective,” or “easiest?” What is the criterion?

Once we know the more specific question (i.e., the criterion—what does “best” mean), there are two general ways of answering that question. One answer is at the level of lots of students with a general need. The other answer is at the level of an individual student with her or his specific version of that need.

That is, there is a general way of answering “what’s best” and an individual way of answering that question.

General

Scientific methods guide comparisons at the general level. When one is concerned about how interventions compare when they are measured according to a specific criterion, the way to determine an answer is to examine a research question. Research questions are fundamental for advancing understanding—they are the drivers for scientific knowledge.

Examples of research questions relevant to special education (where “Y-ing” is some outcome of importance to teachers, parents, and kids themselves):

  • Whether Method M increases Y-ing more than Method N increases Y-ing.

  • Does Method R produce higher levels of Y-ing than Method S under some specific (e.g., “inclusion”) circumstances but different outcomes under different circumstances (“self-contaned”)?

  • Which method—D, J, #, or Q—leads to more students achieving mastery in Y-ing?

When Y-ing is some objectively measurable action (reading words aloud, asking for help, writing answers to arithmetic computation facts, writing an outline for an essay, speaking words at a “normal” volume), then we can create otherwise consistent conditions and compare Methods M and N (and others).

Often, one can find many studies of comparisons of Method M or Method N. Reading those studies (and, especially, integragting them in a systematic way) is expecially important. These exercises are called “integrative literature reviews.” SpedTalk will provide more about literature reviews in subsequent posts. For now, we focus on methods for individual experimental studies.

There are two widely accepted ways to conduct experiments that compare interventions. A team of scientists could use (a) group-contrast or (b) single-case experimental methods. Here’s a summation of each (SpedTalk will provide more details about understanding research methods in other posts):

Group-contrast Methods

In good group-contrast studies, researchers randomly assign students to either Method M or N. They find 100s (1000s, ideally) of comparable students who look like students throughout the land (i.e., are “a representative sample”). They metaphorically, flip a coin to decide whether each student will get Method M or Method N.

There are lots of variants on the coin flip, of course. The coin might be flipped for each of several students in the same classroom; it might be flipped for each classroom; it might even be flipped for each school or local education agency. These are technical decisions made by researchers, and there are considerable debates about what is the right way to make them…but I hope you get the general idea of “random assignment.”

In the simplist form of this research method, the researchers have teachers give the “heads” instruction according to Method M for a while and, at the same time, they have other teachers give the “tails” instruction according to Method N. After an equal period of instruction, the researchers measure all of the students’ performance on a trustworthy measure of Y-ing. The reseachers then examine the average scores on Y-ing for students who got Method M and the average score of those who got Method N. On average, are the students who got Method M higher or lower than those who got Method N? If the average is ‘better’” (i.e, in a more desireable direction ) for one group than the other, then we have a winner!

So, imagine 50 classrooms where all the teachers use Method M and another 50 classrooms (asigned by coin flip) where all the teachers use Method N. Assume that the teachers all implement the methods with equal precision and enthusiam (i.e., what researchers call “fidelity”). Which method “wins?” If the average students in one group of classrooms (the Method M rooms) have better scores on Y-ing than the average in the other group (Method N rooms), that sounds like a winner!

(To be sure, there are possible concerns about these outcomes. Did some groups of students (e.g., girls; those who were low performers at the beginning; some of a specific ethinic group) have better (or worse) outcomes? These are important questions, and they go under the heading “what works for whom under what circumstances.” In future SpedTalk posts we shall examine them. For now, though, please hold them in abayence.

Researchers can use many statistical methods—fancy or simple—to compare the average scores of students who got Method M or N. Readers who are interested in variations about how to make those comparisons should probably pursue graduate studies! (And, remember, SpedTalk will delve deeper into these technical matters in additional posts.)

(b) Single-case methods

Single-subject or single-case methods are an alternative to group-contrast methods. Both can provide compelling evidence of methods’ effectiveness.

In single-subject methods, researchers may randomly assign students to one or another method (i.e., intervention), but they do not compare a group of students who get one method to a different group of students who get the other method. Instead, each student gets both methods and the comparison is between the level of Y-ing for a student during the time when Method M is in effect versus the level of Y-ing for a student during the time when Method N is in effect.

There are lots of ways that researchers can compare Methods M and N using single-subject designs (“reversal” or “ABAB”; “multiple-baseline” and variants; “multiple treatments” and variants). The fundamental consideration is whether, Method M (or N; either is too often called “baseline”) leads to a certain level of Y-ing and Method N (whichever is too often called “treatment”) leads to a lower or higher level of Y-ing. That is, if a student’s level of Y-ing is consistently higher when her teacher is using Method M than it is when that teacher is using Method N, then (assuming higher Y-ing is a diserable outcome), Method M wins.

Now, might that “win” for Method M just be because “it worked” for Sally? Maybe! Singled-subject resarchers test the results that they obtain from one study (the one with Sally, Pablo, and Pete, for example) by conducting very similar studies with other students in other situations with other teachers, and etc. The logic is pretty simple when single-subject researhers replicate the findings, it must be because Method M or Method N is more effective. If every study shows that this horse has a heart, then it must be true that horses have hearts.

(c) Comparison

Note that neither research method (group or single-case) is necessarily better than the other. Scholars debate this question freuquently. Both methods provide an explicit comparison of the methods (M and N in this case). We want to know how those methods (models of pickup trucks?) compare on the variable of interest (criterion of safety, speed, comfort?), so we have to compare them on a trustworthy measure of that outcome.

There is a more important point, though. It has to do with the effects (benefits; drawbacks) of an intervention.

What intervention is best? I don’t know. I think it depends on the goals of the person asking the question about the “best” intervention.

Individual

If parents and educators know the results from multiple scientific comparisons of different methods, they can make an informed guess about what should be helpful for an individual student. Going with a method that has a proven track record is a good bet. If a method has been shown to improve an outcomes for large numbers of students or has been repeatedly shown to improve outcomes in study after study of individual students, then that method should be a good bet for improving that outcome for Sally!

But those of us who know students with disabilities recognize that they are wondrously unique individuals. What "works" for most students may not work for Juan. So, in special education, parents and teachers (and administrators) need to focus explicitly on each individual student.

In this way, questions about interventions for students with disabilities lead almost directly to the very heart of special education: The Individualized Education Program or IEP. IEPs require the planning team to identify goals and objectives--that is, to specify the criterion for success--and to describe how educators and parents will know if the program is succeeding--how to measure progress toward achieving those goals and objectives.

Parents and educators may not know what's best for all students with autism | learning disabilities | intellectual disability | behavior disorders | visual impairments | etc. But we jolly-well can state what we want for Sally (Juan or Liz or Georgia or Petey) and how we'll know if we're getting improvement on that criterion.

So, if teachers, parents, and administrators want to know what intervention is best, they should specify the criteria for comparing methods, consult the existing research to learn what has been found to be more effective in scientifically solid comparisons, and then implement those interventions according to an IEP.

If one wants to buy a new truck, maybe an Individualized Truck-Purchase Plan would be a good place to start.