Thursday, March 5, 2009

Taxonomies of Experiment I: David Sweatt

Experiments are of course a major source of epistemic justification for theories in the special sciences. Different kinds of experiments provide different kinds of evidence. I'll post a few examples of taxonomies of experiment that have been offered by neuroscientists-- one that has been developed by Carl Craver (a philosopher of neuroscience at Washington University in St Louis), and one that has been developed by Alcino Silva, myself, and John Bickle.

This first taxonomy is offered by David Sweatt (pronounced "swet"), an outstanding neuroscientist at the University of Alabama at Birmingham (UAB). David Sweatt has been a pioneer in the molecular and cellular neuroscience of cognition. The following is my transcription of his discussion from his book, so expect some outside references to occur in the passage.

quoted excerpt from David J. Sweatt (2003) The Mechanisms of Memory. Elsevier Press, Boston.

"In general there are four basic types of experiments that any scientist can perform. I refer to them as block, measure, mimic, and determine experiments. I have found this categorization a useful mnemonic device throughout my career as a scientist, and, at the risk of sounding overly pedantic, I strongly encourage any young scientist who reads this book to incorporate them into their thinking about experimental design. For example, every time I write or review a paper I ask whether the investigation has included all these different types of experiments. Especially when writing or reviewing grant applications, where multiyear projects are proposed to test a hypothesis comprehensively, I cross-check myself and others on whether all of these approaches (if technically possible) have been applied to the problem at hand. It is important because what we do as scientists is test hypotheses, and the testing of any hypothesis is much stronger if a variety of independent lines of evidence are available to support the conclusions reached.

What follows is a brief description of each of these four types of experiments.

The determine "experiment" is not really an experiment at all. The determine approach is to perform a basic characterization of the system or molecule at hand independent of any experimental manipulation whatsoever. Examples of this type of pursuit are determining the amino acid sequence of a protein, sequencing a genome, determining the crystal structure of an enzyme, or determining the structure of the DNA double helix. Determinations of this sort are not experiments in that no manipulation of the system is attempted--to do an experiment you tweak the system to see what happens. If you mutate a residue in a protein and see what effect that has on the structure, then you have done an experiment. The basic determination of the structure is not an experiment in and of itself.

Determintations are some of the most satisfying laboratory pursuits to undertake because these are the rare types of studies where definitive data can be obtained. An amino acid sequence is what it is--you get to use unambiguous words like "identical" (versus indistinguishable or similar) and "determined" (versus concluded or inferred) when describing gene and amino acid sequences. There's slightly more ambiguity in determining protein structures and anatomical structures, but in general this pales in comparison to the ambiguity of a conclusion made on the basis of an experimental manipulation. The down side of determinations is that, as a practical matter, they are viewed as boring unless they involve lots of expensive equipment. It's very difficult to get a grant review study section to recommend approval of a basic anatomical characterization, for example, because no experimental testing of a hypothesis is involved. In modern biomedical research, hypothesis testing is de rigueur. In rodent behavioral systems, which are the topic of this chapter, most of the basic behavioral characterization has alread been done. However, there is a growing recognition that more sophisticated and detailed basic behavioral characterizations, and the devleopment of new rodent behavioral models for human mental disorders, is necessary for the next stage of progress in this field.

Block, measure, and mimic are experiments, and they are all specific types of approaches to test different predictions of a hypothesis. For the following discussion we will take the simple case of testing the hypothesis "A causes C by activating B" (see Figure 13).

The mimic experiment tests the prediction that "if B causes C, then if I activate B artificially I should see C happen as a result." An example that we will return to later is: if I hypothesize that a particular protein kinase causes synaptic potentiation, then applying a drug that activates that protein kinase should elicit synaptic potentiation.

The mimic terminology arises from the fact that you are trying to mimic with a drug (etc.) an effect that occurs with some other stimulus, potentiation-inducing synaptic stimulation in this example. The principal limitation of the mimic experiment is that B may be able to cause C but that in reality A acts independently of B to cause the same effect. B causing C and A causing C may be true, true, and unrelated.

At the current state of understanding and experimental sophistication, mimic experiments are just about impossible to execute in the context of mammalian learning and memory. This is because an enormous amount of fundamental understanding of the system is necessary, along with the capacity for very subtle manipulation, in order for the experiment to work. For example, suppose I hypothesize that synaptic potentiation underlies learning. In theory, the mimic experiment is to put an electrode in the brain, cause synaptic potentiation, and then the animal will have an altered behavior identical to that caused by a training session. Of course, doing this experiment requires that I know exactly which synapses to potentiate so that I can selectively acheive the right behavioral output--this is beyond the level of understanding for essentially all mammalian behaviors at this point.

The measure experiment tests the prediction that "A should cause activation of B." Using our example of kinases in synaptic potentiation, the measure experiment should cause an increase in the activity of the kinase. This is, of course, determined by measuring the activity of the kinase as directly as possible, hence the measure terminology. The measure experiment has been applied in a variety of different ways in the memory field, ways that we will discuss at various points throughout the book including looking for anatomical, physiologic, and molecular changes in the nervous system in association with learning. The principal theoretical limitation of the measure experiment is that it is correlative. One can show that A causes activation of B, but that does not demonstrate that activation of B is necessary for C to occur.

Which brings us to the block experiment. The block experiment tests the prediction that "if I eliminate B, then A should not be able to cause C." In our working example, this means that a kinase inhibitor should block the ability of the potentiating stimulus to cause potentiation. At present, the vast majority of investigations into mechanisms of memory involve this approach, and we will make many references to this type of experiment throughout the book. Specific examples include anatomical lesions, drug infusion studies, and genetic manipulations. The principal theoretical limitation of the block experiment is that it does not distinguish whether activation of B is necessary for C, versus whether the activity of B is necessary for C. For example, suppose that B provides some tonic effect on C that is necessary for it to occur. Inhibiting B will block the production of effect C when in fact A never has any effect on B whatsoever. In behavioral terms for learning experiments, this is referred to as a performance deficit-- the animal is simply unable to execute the behavioral read-out necessary to exhibit the fact that they have learned.

In summary, then, the mimic experiment tests sufficiency, the block experiment tests necessity, and the measure experiment tests whether the event does in fact occur. Each type of experiment has its strengths and weaknesses. Positive outcomes in testing each of these three predictions for any hypothesis makes for clear, strong support of the hypothesis." (Sweatt 2003, p. 45-46)