What can we do?

Research in education is a most worthy goal. There is just one problem. In the US at least, it has become grossly politicized over the past few decades

Given the many interpretations a single set of data can yield, it is obvious that beauty lies in the eye of its political benefactor. The same is true of the ugly, the pleasant, the fearsome. What to do? How can the extreme points of view be brought together in common purpose? Are common denominators enough? Can we try and see? Can mutual goodwill prevail as all religions seem to teach? Have we tried? For what educational goal is there broad appeal? Who is looking? Where are the answers?

Of one thing we can be sure. Truth can be a way out--if it is itself not politicized. Urban VIII is a prime example of denial of truth, that Copernicus was right. It is no different today except that politicians now stand in with certain ecclesiastics in opposing science, and education fills in for the theory of Copernicus. The 17th century was a polarized world, more so than today. So we can take heart and try. If the common goal truly is education in and of itself, then research on how best to educate is in everyone’s interest. At least it should be.

Social research is of the softest kind--it is difficult, often impossible, to see small, but real, effects in seas of noisy data. So it behooves all concerned to look for and heed only the strongest signals, those that cannot be denied or misinterpreted. Urban refused to view “Galileo’s moons” circling Jupiter. We cannot afford to make that same mistake for the sake of politics, religion or economics, three modern American hot buttons.

Having said that, it is certain that effective education must in the end cost less--simply because it is efficient. Equally certain, effective education will evolve in progressive ways. Together, these features should appeal to both the fiscal conservatives and the social liberals. Since there is something here for everyone, there is common ground in how we look for and implement the fruits of research. Done properly, research into education will have something in it for everyone.

Sound research may not be as difficult as it sounds. Although the methodology of science is obscure to many, we know a great many folks who have the intuition of the scientist. They realize instinctively that an effect may have more than one cause, or even that one cause can influence the strength of another cause to bring about a given effect. The only thing that separates a capable scientist from the rest of us is that the scientist has learned how to put numbers on the strength of a cause while providing an error bar for the resulting effect under study.

In education, the effect may have several dimensions in its output. For example: Knowledge (information in and of itself), Leadership (the “go-to” people), Capability (highly-honed skill sets), Communicators (media workers, conflict resolvers), Visionaries (tomorrow’s researchers). On the other hand, those being educated are a varied lot. On one extreme are those quite capable of fully educating themselves--we have known a few. On the opposite pole are those who require special help, just to reach basic competency. Then there are the dyslexics along with sociopaths and their kin who require still different procedures. Any of these may have great talent, yet be lost to both society and to themselves under the present system. All this is not to say we have not made progress. We most assuredly have, century over century. But we are still far from being where we could be if we could only fit each child (or adult) into an educational system that is right for him/her.

The concept that “one size fits all” only applies to those of us who happen to be near the median of all people in their many personal traits. That group is not likely to be more than about 2/3 of all of us. What to do with the rest is already partly known. Special-education teachers know their turf well. But what about the other pole, the 5% or so who are gifted? Are they as well known? Their principle need is general guidance with feedback more than classroom work. The system itself must be broadly accommodating. That is an ideal system to be worked toward. How can progress be made? Where do we begin?

First the cohort, the people whose response to various methodologies we will measure. We propose that the middle folks, near the average, the norm so to speak be our initial cohort. They will be the journey people of tomorrow. The so-called average person still has far more capability than we now develop in our system. These are the folks who would provide data to be learned from or discarded. Some of that data could read over to the less typical folks for later testing out.

Now the conditions, and this is where things get a little sticky. This is the arena that has been politicized by all interested players. Nevertheless, to be scientific, all variables known or strongly suspected of being able to influence scholastic performance must be controlled for. Social strata is one well-known example. Family wealth might be another. So also for family functionality. Teachers themselves are perhaps the most critical variable. Controlling "for" does not mean" excluding," it means including. More on this below.

Variables to be included bring up the issue of sample size and its cost-benefit. Large samples add certainty to any analysis, but they cost more to follow and analyze. Samples too small are in danger of providing false leads, missing real leads, or becoming mere anecdotes. Sample size is especially critical in the social sciences. One reason is that there is much variation within each individual performance. That variation is the bottom-line against which all other variables must be tested for effect. The other variables have their problems as well. An individual’s family history is another well-known variable affecting performance. But how does one quantify it? Other variable are even more vague. The upshot is that it is rare in social science to be able to show that any given variable accounts for even 20% of the variation in responses observed. Most “significant” results are below this level. With very large sample sizes, one might find a significant effect that only brings about a 5% response. These are hardly worth chasing.

It is mainly for this reason, that effective research in education, must look only for the large variables, and hone them. Some, like the teacher effect, are already well known and large. Personality plays a role here. Students who are most motivated achieve the most. Teachers vary considerably in their ability to motivate. An example of how one variable can affect another comes up here. Consider two teachers with similar personality traits, equal in ability to motivate, teaching the same subject. The one who has mastered the material being taught will motivate better than the less well-prepared one. This statement comes from long personal experience in trying to motivate others. To that we add observations of others doing the motivating. So we ask ourselves, which is more important, the skill set or the ability to motivate? And how do they interact--if they do? Our anecdotal experience leans toward motivation and there is some logical basis. But anecdotes are not proof.

Skill sets and motivational abilities are important questions to education as they dwell on what kind of teachers are best at educating others. Both the schools of education and the school principles are interested in this one, or should be. So also for all strata in a society, its left and right.

Back to our scenario. How can we assess which of these features has the stronger effect? And how certain can we be of what we find? This is where experimental design comes in. There are two aspects. Intuition driving what to look for, and then figuring out how to find it and decide if it is real or not. The latter is a highly developed science and has rigid rules. But it is our experience that most people already know that all variables need to be controlled. So we are halfway home on that: We can select a sample, a cohort of people as nearly alike as possible and determine how well they learn under the two conditions mentioned above, skill set and motivational ability. So good,so far.

Now we think, “well we divide the cohort in two, have two teachers that fill the experimental conditions each teach half of the cohort. Then we test each half for what they have learned after some number of sessions.” What could be simpler? In fact, there are a great many studies out there less rigorous than this one. But can we be criticized? We got answers to our question did we not? “Yes of course, but what do they mean?” a critic will ask. “The difference we saw is what the data mean.” we reply. The critic presses on, "Well I am not sure; how did you divide our cohort?” This is a good question, for unless class assignments were truly random, where each person in the cohort had an equal chance of being assigned to either group, we may have a biased result. Overlooking things like this has discredited many studies. For this very reason it is common practice for a research planner to run his/her experimental design by other people before implementing a study. Of course, there are a myriad of other possible problems in a research design. But the simplest of all can still miss an important feature, as just illustrated. Furthermore, cohort from Indiana are not like those in California.

How can the average person sort out good research from bad?

• Has it stood the test of time? The apparent "instant fix" must always be suspect. Yet politicians compulsively lead us toward one instant fix after the other. They do it with ease because our desires for an instant fix are insatiable.
• Which among the many possibilities is right? The real world in social science usually turns out to have many "right answers," like bricks in a wall. The mosaic is what we are looking for.
• Has true cause and effect been shown? If this is not true, then no predictions are possible. Mere correlation does not establish a cause and effect relationship. There must be a logical connection. A human is smarter than a dog starts out as a correlation. When brain size and function are added to the equation, what was correlation becomes cause and effect. The connection has theory behind it; theory that predicts humans are always smarter than dogs,

To summarize and add a few vital features:

• Start with the variables believed to work and plan to define their 1) true strength of effect and 2) their degree of certainty.
• Select cohorts that are similar in all ways that matter to the research.
• Divide the cohort randomly into however many groups you wish to study, making sure there are enough individuals in each group to produce meaningful data.
• Establish their starting knowledge with the same test instrument for all under the same conditions for all.
• Perform the teaching by methods called for.
• Determine how much of the new material they learned, again under conditions the same for all.
• Report all data and describe it in terms of the differences, trends or effects searched for. List what was controlled for as well as what variables were tested.
• Analyze the data for what it means and make your results known to others.

This is greatly simplified, but the issues are basically simple, even if the design is complex. It all comes down to integrity of the data and its rendition. Nothing else will do. Research in education must be free of bias, employ controls, and use large enough samples that the data can mean something beyond the vagaries of blind selection.