What If Plato Took Surveys? Thoughts about Philosophy Experiments.

The movement called Experimental Philosophy (‘x-Phi’) has now passed its tenth anniversary. Its central insight is compelling: When an argument hinges on accepting certain ‘facts’ about human perception, knowledge, or judging, the evoking of relevant intuitions by thought experiments is intended to make those facts seem obvious. But these intuitions may not be shared universally. Experimentalists propose testing claims that traditionally were intuition-based using real experiments, with real samples. Demanding that empirical claims be empirically supported is certainly reasonable; though experiments are not necessarily the only means available. When experiments are conducted, adequately interpreting their results requires understanding the study’s design (and possibly flaws) that produced them. Experiment-based reports should document the design clearly. If Plato, writing his Meno, replaced accounts of Socrates demonstrating geometry to a slave-boy, with a survey of 100 real boys—some grasping his demonstrations, others not—what conclusions could be reached? Before answering, the reader needs details on key design questions, including (among others): (a) What population are these samples intended to represent? (E.g., ‘all slave boys’?; ‘math-ignorant people’?; ‘everyone’?) (b) What statistical tests were conducted, on what assumptions? (c) How was ‘significance’ of results determined? (d) Was the test instrument’s validity established? For readers wishing to explore x-Phi’s potentials, as contributors or as interpreters of their findings, this paper offers some cautionary considerations. Throughout their literature, examples can be found casting doubt on some experimentalists’ findings, due to design-related issues. Increased awareness of methodological questions would tighten the x-phi literature, going forwar