[ Read the second part ]
The Critique
A predominant problem of the reviewed experiments is the sampling. Sheldrake & Smart use rather vague terms in their report; it is explained that the phenomena studied is the most common example of telepathy in the modern world and you get the impression that the ability to sense who is calling before you answer is general. But does the study sample really represent the general public? The receivers who responded to the advertisment reasonably considers themselves to have telepathic ability, a notion stronger than a general supposition. Futhermore, it is reasonable that the callers nominated by the receivers to some extent shares this notion and also thinks the receiver has the supposed ability. The sample must therefore be considered representative of a population of what is usually called "high scoring subjects" rather than the general public. This narrowing down is, however, not done by Sheldrake & Smart, an error also noted by Schmidt, Müller, & Walach (2004).
The snowball sampling of callers is motivated by the theory that telepathy occurs between people belonging to the same social group. But in a group of relatives and friends, there is so much more than an alleged telepathic connection. There is affinity, loyalty, shared values, group pressure, and social obligations. A systematic error to be considered in all psychological research is the "good-subject tendency," i.e. the tendency of experimental participants to act according to what they think the experimenter wants. In the classic Milgram experiments, two thirds of the participants were willing to administer dangerous electroshocks when told to do so by a "professor" in charge, even when the victims begged to be released or in the end responded to the shocks only with tormented screams (Milgram, 2004). Dissimulation as well as lies can be part of participant strategy to achieve what is believed to be the experimenter's goal. In an experiment with members of social groups participating together, with an established affinity, it is imperative to be aware that this type of participant tendency can manifest itself in both implicit and explicit cooperation within the participant groups. In an experiment on telepathy it is of the utmost importance that both verbal and non-verbal communication can be ruled out as cause of a measured effect, especially when participants are allowed to act within the frames of existing social bonds and forms of communication perhaps unknown to the experimenter. Are Sheldrake's & Smart's experiments controlled in these respects? Does the experimental design allow other forms of communication than telepathy? Here are some suggestions of confounding variables:
Positive interpretation
All reported methods allow what can be called "positive interpretation." The experimenter is actually totally unaware of what is being said during the calls between receiver and caller. The receiver reports his or her guess to the experimenter first after the call and any caller confirmation is also done after the call. The experiments thus lack any control for interpretations in line with the following:
- Uhm, I'm guessing Frank. Is it Frank?
- No, it's Mary. Sorry.
- Oh, I thought of you first but then I changed my mind.
- You did? Well, you were right then, from the beginning.
- Yeah, typical...
- But let's report you were right. The idea is to follow your intuition, isn't it?
- I suppose... But can I do that?
- Of course you can. It's not cheating since you thought of me first.
- I guess... Okay, I'll report I was right.
Both receiver and caller then reports that the guess was right, even though it de facto was wrong - the receiver might even have thought of all callers. None of Sheldrake's & Smart's methods is protected against this threat to internal validity. This type of interpretation is also more likely to emerge within a social group than between strangers who doesn't know each other. Thus, the results of the study don't exclude this kind of error; the fact that no telepathic connection beyond chance has been measured between receiver and unknown caller can be explained by the lack of social bond that permits this type of "agreement", rather than lack of a shared "morphic field." And it doesn't take many such instances of positive interpretation to significantly affect the data.
Caller number identification
The report carefully describes what kind of die was used in the random selection of caller and calling time but there is no description of what kind of telephones were used by the receiver. It would have been appropriate that the equipment used had been accounted for. Was it an older type of phone or a more modern with a display? Does the receiver have caller number identification service and if not, can this be verified by the receiver's service provider? None of this is accounted for in the report.
Cell phone/SMS
Since both receiver and caller are sitting all by themselves in their homes, there is no possibility to control possible verbal or text communication by cell phone. To exclude such communication, careful monitoring of both receiver and caller is required - a receiver may even have a cell phone set to "silent" kept close to the body, and be directed by agreed upon vibration signals.
Sensory leakage
Even if the calls had been controlled, Schmidt, Müller, & Walach (2004) notes the possibility of sensory leakage as the receiver might apprehend cues from different sounds from the caller and his or her environment. This threat is eliminated if the receiver has to make a guess before answering. But as already noted, the Sheldrake & Smart experiments lack control of the calls altogether.
In Sheldrake's & Smart's own discussion on errors, mortality and how it might have affected the statistical analysis is considered. But the mortality itself is not. In both experimental series, the mortality is 57%, i.e. more than half of the receivers dropped out before completing the ten trials. For an experimental study in which the participants have been informed in detail about procedure and in addition gets compensated, 57% is a whopping number. Sheldrake & Smart provides the following explanation:
They withdrew for a variety of reasons, most commonly because they could not persuade all 4 callers to agree to be available at the same times. (Sheldrake & Smart, 2003a).
In the discussion on errors, this is somewhat elaborated on:
In fact most participants who stopped did so because their callers were unable or unwilling to continue. (Sheldrake & Smart, 2003a).
The receivers, who at one point decided to participate, were thus unable or unwilling to complete the experiment. Why? Sheldrake & Smart claims that it usually was due to the four caller not being able to participate at the same time. What were the reasons that cannot be sorted under "usually"? Is it possible that some of the receivers saw no point in continuing because of weaknesses in the experimental design? Sheldrake & Smart doesn't say.
It should be noted that the vague formulations regarding mortality allows for it all to be explained by unwillingness to complete a scientific experiment because it was experienced to be unscientific or otherwise not worth completing, despite compensation. Even if the participants knew each other well in the groups, they were not familiar with participants in the other groups. If you dropped out from the experiment because you considered it meaningless you might still get the impression from Sheldrake's & Smart's report that most of the others left the experiment due to time factors.
Sheldrake & Smart use initials for the receivers in their report. An account of the mortality, listing the reasons for dropping out, would have been possible considering both space and practicality. In light of the methodological weaknesses of the experiment, such an account should have been in the report.
In the discussion on errors, Sheldrake & Smart dismisses cheating for three reasons: (1) It is unlikely that a majority of the participants would have cheated and had that been the case, the results would have been different, (2) Sheldrake & Smart themselves did not cheat in the preliminary experiment, nor did the unknown callers in the second experimental series, and (3) the tests in a completely different experiment was videotaped (Sheldrake & Smart, 2003b).
It is certainly unlikely that a majority of participants would cheat, had the sample been random and representative of the general public, but that is not the case in the Sheldrake & Smart experiments. Instead, the researchers have a non-random sample of people who think they have a telepathic ability or that the receiver does. It is not unlikely that a majority of such a sample considers it reasonable to somewhat adjust guesses in line with the "positive interpretation" principle described above - even if it means cheating in a strict experimental sense, it doesn't have to have that meaning to the participants. Sheldrake & Smart also presumes that cheating would have rendered even better results than was the case, thus ignoring the possibilty of participants assessing that major "adjustments" would be suspicious. Since the experimenters beforehand informed the participants of the procedure, they certainly disclosed information about what outcome would be expected by chance and thus hinted what results would be enough to be significant. It would be foolish - and revealing - to achieve more than necessary.
That Sheldrake & Smart themselves didn't cheat doesn't say much, unfortunately. The fact that Sheldrake is looking for support for his very controversial theory on morphogenetic fields makes him more than inclined to "interpret positively". Smart was alredy convinced she had telepathic connection to her dog (Sheldrake & Smart, 2000) and must be considered just as inclined as Sheldrake to adjust results. Furthermore, it is rather remarkable that experimenters double as participants in experiments.
The argument that unknown callers did not cheat (their results are at chance level) is only valid if the only possibility of cheating is the one suggested by Sheldrake & Smart. As is evident from the above, that is not the case.
The experiment videotaped in another study (Sheldrake & Smart, 2003b) cannot guarantee that cheating was not used in the reviewed study. It is of course ridiculous to suggest such a thing. If you look at both studies you find that one woman, Sue Hawksley, scored 63% when her mother was calling in the reviewed study, but only 27% when her mother was calling in the videotaped experiments. The same woman scores at chance level when close friends are calling in the reviewed study, but 63% and 45% in the videotaped experiments. It seems highly unlikely that an alleged telepathic connection with the mother disappears entirely when someone turns on a videocamera, and that telepathic connection with close friends then suddenly emerges. A reasonable explanation would instead be that the mother is comfortable with "positive interpretation", which is possible in the reviewed study, while cell phone or caller number presentation, which are possible in the videotaped experiments, are more fitting cue tools for the close friends.
A fundamental experimental flaw is the fact that the research method is changed three times during the experiments. It indicates that the design was weak and not thoroughly thought out from the beginning. The fact that Sheldrake & Smart so freely reports this suggests a lack of fundamental insights into research methods.
Sheldrake (2006) claims that the experiments have been replicated but the study he refers to, Lobach & Bierman (2004), suffers from similar flaws as the one reviewed here. Another study, in which errors were eliminated and stricter controls adopted, reported results that would have been expected by chance (Schmidt, Müller, & Walach, 2004).
To sum up, no inference regarding the existence of telepathy can be made on basis of this study. As it has been reported by Sheldrake & Smart, it suffers from methodological flaws so severe that it must be considered worthless. The researchers present a seeming representability that doesn't exist och appear totally unaware of fundamental psychological dispositions, in themselves and in the participants, that might be of importance to the experiment. The design is very weak and sensitive to several confounds that may affect the results.
Next, a concluding post. Stay tuned.
References
Lobach, E., & Bierman, D. J., (2004). Who’s Calling at This Hour? Local Sidereal Time and Telephone Telepathy. Report presented at The Parapsychological Association Convention, Wienna, Austria.
Milgram, S. (2004). Obedience to authority: The unique experiment that challenged human nature. New York: Perennial.
Schmidt, S., Müller, S., & Walach, H., (2004). Do You Know Who is on the Phone? Replication of an Experiment on Telephone Telepathy. Report presented at The Parapsychological Association Convention, Wien, Österrike.
Sheldrake, R., (2006). In Conversation on Abc Radio National – Rupert Sheldrake [www dokument]. URL http://www.abc.net.au/rn/inconversation ... 754367.htm.
Sheldrake, R., & Smart, P., (2000). A Dog That Seems To Know When His Owner is Coming Home: Videotaped Experiments and Observations. Journal of Scientific Exploration, 14, 233-255. URL http://www.sheldrake.org/Articles&Paper ... video.html
Sheldrake, R., & Smart, P., (2003a). Experimental Tests For Telephone Telepathy. Journal of the Society for Psychological Research, 67, 184–199. URL http://www.sheldrake.org/Articles&Paper ... _tests.pdf
Sheldrake, R., & Smart, P., (2003b). Videotaped Experiments on Telephone Telepathy. Journal of Parapsychology, 67, 187–206. URL http://www.sheldrake.org/Articles&Paper ... video.html
Posts
No comments:
Post a Comment