This article is the keynote speech given July 18, 1997 at the
1997 Visitor Studies Association (VSA)
conference in Birmingham, Alabama. It was published in the 1998
proceedings from that conference and is provided here by special
permission from VSA.
Let me start this lecture by thanking Stephen Bitgood and the Visitor Studies Association for inviting me to address this gala 10th anniversary gathering. I also want to thank the National Science Foundation for their support of my work. A grant from the Informal Science Program in 1995 enabled me to devote several months to thinking and writing about ideas I will discuss today. Without this support, I would not have been able to go as far as I have. Progress in the field of visitor studies during this decade has been impressive. What was once a small, fringe segment of the museum community has grown into a recognized, solid component of both museum professional work and social science research. We have accomplished a great deal. But if we are to continue progress on both fronts, we need not only to undertake more visitor studies, but also to be more reflective about our work. I want to contribute to this latter effort today.
The title of this talk provides a convenient visual metaphor and indicates the general outline of my thinking. I believe that there are at least two contrasting ways to view both educational and research theory as it applies to our work. These two, which I have described metaphorically as "maze" and "web," reflect different world views, or world hypotheses, about the nature of each domain.
Mazes offer many possibilities to get lost and many paths that lead to dead ends. But most important, a maze has a start and a finish, a correct solution (usually only one, although several are theoretically possible) and multiple paths that are definitely incorrect.
Webs have quite different characteristics. Spider webs, used frequently as models for multi-path theories, do not have correct and incorrect directions, and are bounded only by the physical limits of their structure. They also differ, depending on the species that spins them. Recently, we have all become familiar with another model that uses the web as its central image, namely, the World Wide Web of the internet. This information connection system, like spider webs, has no right or wrong paths through it. Each user develops his or her own connections; each application can result in successful information transfer following multiple different paths, links, or nodes, and there is no start or finish, only multiple entries and multiple possible fruitful conclusions.
These characteristics of mazes and webs can be applied to both education and research theories. In the remaining time, I plan first to outline the distinction in education between constructivism and traditional theory, then do the same for research theory by discussing distinctions between naturalistic and experimental-design approaches, pointing out that in each domain one position is more web-like while the other is more maze-like. Finally, I will argue that there is a relationship between theoretical positions that share web or maze features in both education and research. For more detailed elaboration of these views than is possible here, I refer you to a longer work on this subject that will be available soon (Hein, in press).
Educational theories can be distinguished by their positions concerning epistemology and learning theory (Hein, 1995, 1996a, 1996b). We generate a theory of education by embracing some view of what it is that people learn, as well as a position on how they learn. Within each of these domains, a continuum of concepts determines our views. Thus, for the content of education one extreme position is that what is learned represents the world as it actually exists; what we learn corresponds to the way the world really is. This view, called "realism" by philosophers, has a long history that can be traced back to Plato. In contrast, an equally venerable position argues that we cannot comprehend the real world (if any such exists), that the subject of our learning resides in human minds; it consists of ideas that we create. This position is called "idealism" in philosophy. In the language of our metaphor, the learner is either a searcher for the correct path in the maze, or the spinner of a web. The two positions, as they relate to museum education, are illustrated in Figure 1.
Similarly, contrasting views concerning learning theory are possible -- the mind as passive recipient of knowledge or the mind as active creator of knowledge. The latter view is primarily a product of 20th century formulations of educational theory, most notably John Dewey's position and the strong influence of developmental and socio-cultural theories of learning. In recent museum literature, Roschelle (1995) has discussed this position in detail. Again, in the language of our metaphor, learning can be considered a method for finding the correct path in the maze or an exploration of the many fruitful paths in a web. We can picture the two extreme views as the ends of a continuum, as illustrated in Figure 2.
A two dimensional map of the four possible educational theories generated by combining these two continua is illustrated in Figure 3. I have labeled the four quadrants as they apply to museums and described them in detail elsewhere (Hein & Alexander, in press, Hein, in press). For the purposes of this talk, I want to focus on just two of the quadrants, those labeled "traditional lecture and text" and "constructivism," since they most sharply illustrate the difference between maze-like and web-like educational theories.
If we espouse traditional educational theory, then the primary focus of our educational effort will be to appropriately organize the "true" nature of the subject to be taught, from the simplest elements to the most complex, so that learners can absorb the knowledge. The emphasis will be on the appropriate structure of the subject that provides the logical, correct path through the maze of content so that learners can obtain the information in the proper order and build increasingly complex understanding of the hierarchically arranged material.
This view of knowledge and learning leads to a linear (and vertical) conception of educational content, whether referring to formal curriculum or exhibitions. The most detailed formulation of such ideas has been provided by Robert Gagné (1977), who developed detailed learning hierarchies for a wide range of subjects to guide educators in constructing educational materials. His theories were the basis for one of the major curriculum efforts of the 1960's, Science a Process Approach, developed by the American Association for the Advancement of Science. For example, he has an elaborate scheme for learning about the conservation of volume, a topic for which Piaget offers a totally contrasting explanation of how it might be learned. In Gagné's formulation, a child must first understand the concepts of "liquids" and "solids" then learn the rule that "liquid may be poured into a container," followed by the rule that "liquids assume the shape of the container" and so on, until this hierarchy of concepts, combined with other hierarchies about area and volume, are combined to reach an understanding of the rule that "volume of liquid is determined by l, w and h." (Gagné, 1968, p. 184)
An educator must have a clear understanding of the hierarchy of concepts and skills required to comprehend a complex idea in order to generate an appropriate path through the educational maze to guide the student to correct understanding. Although most practicing educators do not develop as elaborate schemes as Gagné did, many do struggle to find the one, true sequence most appropriate to guide learners through any subject.
This approach to subject matter stands in sharp contrast to the constructivist view, which begins with learners' interests and prior knowledge and develops curriculum based on where these starting points may lead. In fact, this alternative approach is often described as building curriculum "webs," in recognition that the way the subject is developed in an educational setting is not dictated by a necessary content structure but opportunistically, in response to the paths that individuals or groups of learners may take as they build their understanding. An example of such a curriculum "web" is provided in Figure 4 (Corwin, Hein, & Levin, 1976). This approach to curriculum does not assume that the subject itself dictates a particular order and leads to a particular outcome, but rather that a combination of prior knowledge, interest, and experience combined with subject matter can lead to a variety of learning outcomes, depending on the interaction between the learner, the subject and the teacher. When applied to museum exhibitions, this model suggests that exhibitions would not be arranged linearly, that exhibit components might offer alternative rather than sequential material, and that visitors would chose among components.
Similar distinctions between maze-like and web-like approaches can be made for classification of mental skills. The familiar learning hierarchies described in Bloom's taxonomy (Bloom, et al., 1956), from knowledge at the lowest level through comprehension, application, analysis, synthesis to, finally, evaluation, implying that there are higher order and lower order mental skills, can be challenged if viewed from a different perspective. (2) (3) Champagne (1990) has argued that any attempt to classify thinking skills as higher order and lower order results in an empty category for the lower order since all mental skills turn out to be complex.
Finally, consider different models for the relationship between learning and teaching. The conception of these two human acts illustrated in Figure 5, that teaching leads directly to learning, is certainly too simplistic. But how we formulate a more realistic and comprehensive model of this relationship depends on whether we adopt a more maze-like or more web-like approach to educational theory. One way to think about the relationship between learning and teaching is to consider that learning is the result of some additive combination of a series of factors, all of which contribute to learning, and each of which can be analyzed and studied separately, as illustrated in Figure 6. The aim of education is to balance appropriately the influence of each factor that influences learning; to find the correct contribution of each to reaching the goal.
In contrast to the formulation in Figure 6, Figure 7 illustrates a social-cultural view of learning favored by some cognitive scientists. They argue that the various components that contribute to learning are interconnected in ways that are too complex to be separated, and should only be studied within the learning-teaching situation. For example, Rogoff (in press, see also Matusov & Rogoff, 1995), has argued that any attempt to separate teaching and learning is artificial.
The point of the discussion above is not to argue for the value of one educational theory over another, but to illustrate that there are different ways of framing educational theories, and that these formulations, based on incompatible models of how the world works, provide distinct bases for practical activity in education.
Research theories, like educational theories, can be formulated along distinct lines. In general, experimental-design approaches see the world as a puzzle that needs to be broken into its component parts and solved, while naturalistic approaches view the situation to be studied as a complex interconnected set of conditions that need to be comprehended and described rather than analyzed. Again, the maze and web analogy is applicable. Visitor studies literature recognizes these two contrasting approaches to evaluation and research, which I label "experimental-design" and "naturalistic" (see Bonner, 1989). (4)
As in the case of educational theories, realistic and idealistic conceptions of knowledge lead to different descriptions of what research should be about. These contrasts are illustrated in Figure 8. Another way to distinguish experimental-design and naturalistic approaches to research is to describe contrasting attributes of the two, illustrated in Figure 9, derived in part from work of Guba (1978, Lincoln & Guba, 1985).
I've presented only a brief summary of the distinctions between maze and web theories of education and research, gleaned from the huge literature, some of it quite vitriolic, which describes them. (5) The heated nature of the debate suggests that when people write about these views they are, indeed, looking at the world from quite different perspectives, and therefore write about the "mistaken" views of those who hold a different conception of how to interpret the world.
Commonalties of Maze and Web Research Theories
Before examining differences between naturalistic and experimental-design approaches to research, it is worthwhile examining some aspects of visitor studies work that are frequently discussed but are not seriously influenced by the differences between research ideologies.
I believe that the differences between naturalistic and experimental-design approaches do not affect efforts to classify visitor studies work as research or evaluation, nor to distinguish between front-end, formative, and summative evaluations. Although the latter statement is probably not controversial, the former (the relationship between research type and the evaluation-research distinction) probably is. Shettel (1991) and Miles (1993) have both argued that research, because it is intended to create new knowledge, requires what I have called experimental-design approaches. I believe this position is based on a circular argument. If the outcomes of research are defined as those resulting from the experimental-design model, then, of course, only that model can lead to new knowledge. But that is precisely the point that is argued by advocates of naturalistic methods. They assert that there are alternative views of the world, which both require alternative methods of inquiry and lead to different kinds of conclusions.
Another generalization which I don't think holds up on careful examination is that older, early 20th century visitor studies consisted exclusively of experimental design studies, while the naturalistic approach is a recent addition to the field. I believe both traditions are as old as visitor studies.
The first visitor study on record includes both quantitative and qualitative information, and illustrates both approaches. Higgins (1884) provided numerical data on various types of visitors to the Liverpool Free Public Museum of Natural History, but also gave us a descriptive paragraph about one category of visitors with no estimate of their relative numbers. He obviously decided that a qualitative approach was more appropriate to describe this group, independent of the percentage of visitors they represented.
One other class, however, deserves distinction; and it is gratifying to mention the admission, year after year, of streams of German and Scandinavian emigrants, who, after seeing their packages piled up at the railway station, seem to pass almost immediately to the Museum or the Gallery of Art. In the midst of the distractions of the most important crisis of their lives, these strong-hearted men and women find time and inclination to increase their stock of knowledge; and, though they are unable to understand the explanatory labels, their conduct strikingly indicates respect towards the institution and its purpose. (Higgins, 1884, pp. 186-7)
Another early study by Gilman (1916), described in detail by Loomis (1987), uses 32 photographs of a hapless museum visitor stretching, crouching, straining, and even climbing on a ladder to illustrate why museum display practices lead to museum fatigue. The study is a classic illustration of naturalistic research. There is no way to formally generalize from the descriptive results (the Boston Museum of Fine Arts in which he carried out his research might be a unique environment), yet Gilman's photographs and accompanying descriptions provide a powerful argument for the need to rethink display practices.
Finally, Alma Wittlin's (1949) splendid study of the effects of changing a gallery to include fewer objects, more illustrations, and more interpretation provides another example of a relatively early visitor study that I would classify as naturalistic. She observed and interviewed visitors and even asked them to draw their reactions to the two types of exhibitions. She concludes that the "New Exhibition" is more effective than the "Old Exhibition" and supports her conclusion with typical naturalistic data, quotations from interviews and other descriptive material.
The long tradition of naturalistic research in museums parallels the equally long tradition of naturalistic research elsewhere in the social sciences. The relatively recent introduction of this approach into the formal education literature, especially in the United States, resulting in frequent debates in Educational Researcher (see for example, Anderson, Reder, & Simon, 1996, 1997, Greeno, 1997), shouldn't blind us to the fact that the issues involved and the debates about methodology in the social sciences go back to the origins of the fields devoted to the study of human activity. In discussing the history of the struggle to change the direction of psychology from a behaviorist effort to one which included a more web-like view of how the mind works, Bruner (1990) reminds us that a major component of this effort was to incorporate a range of methodologies from disciplines usually associated with more naturalistic research approaches into the field.
Now let me tell you what I and my friends thought the [cognitive] revolution was all about back in the late 1950's. It was, we thought, an all-out effort to establish meaning as the central concept of psychology -- not stimuli and responses, or overtly observable behavior, not biological drives and their transformation, but meaning. It was not a revolution against behaviorism with the aim of transforming behaviorism into a better way of pursuing psychology by adding a little mentalism to it. Edward Tolman had done that to little avail . . . The cognitive revolution, as originally conceived virtually required that psychology join forces with anthropology and linguistics, philosophy and history, even with the discipline of law. (Bruner, 1990, pp. 2-3)
Contrasts between what Tinbergen (1974), in his Nobel prize acceptance speech arguing for acceptance of more diverse methodologies in science, called "the science of watching and wondering" and experimental-design approaches have a long tradition (Hein, 1976).
Distinctions Between Naturalistic and Experimental-Design Research
Perhaps the most powerful quality of naturalistic research when carried out by its confirmed practitioners, is the conviction that it represents a different way of thinking about research. While this difference may be hard to quantify, it often shows up in how the approach to research is discussed. Proponents of the argument that the two can be easily combined and represent interchangeable stages of work, sometimes justify naturalistic research for its value in carrying out preliminary exploration -- it's useful for pilot studies until the parameters are identified -- or as a convenience in situations where the individual components are too complex to be readily identified and the resources available preclude a more "rigorous" experimental approach. Advocates of naturalistic research don't use such apologies. They see it as a unique and independent form of inquiry, justified on its own merits and leading to its own types of conclusions.
An example of an effort to minimize the difference and treat it as only a technical concern is found in a recent article by Walberg, et al. (1997). The authors suggest that the "subjective" nature of naturalistic research and the tedious and labor intensive tasks of analyzing qualitative data can be avoided by using newer computer programs that will automatically sort text and combinations of text by frequency. Thus, the themes that naturalistic researchers find in data will "emerge" not through fallible, human analysis but by simply recognizing the most frequently used words or word combinations in respondents' answers to open ended questions. Walberg, et al. argue that computer generated word frequency and word-combination frequency allows "precisely repeatable analysis without a priori frameworks and without tedious and subjective categorization of passages" (Walberg, et al., 1997, p. 55), although they admit that, "as in the case previous content analysis . . . subjective interpretation is required after the analysis" (p. 57). This approach denies the essential quality of naturalistic research by treating the judgment and analysis of the researcher only as a problem without also considering it a positive opportunity.
The sections below briefly summarize some differences between the two approaches. Again, there is a voluminous literature discussing these points. For a good summation of most of the arguments in favor of naturalistic research, I refer you to Denzin and Lincoln (1994).
Reliability and Validity
The two forms of research place different emphases on reliability and validity. In order to carry out experimental-design research, much is sacrificed in the effort to assure that the research protocol is reliable, in other words, that, if repeated, the same results would be obtained. A frequent criticism of such work is that the effort to establish a laboratory-type setting in the interest of increasing reliability modifies the situation studied to such an extent that results need translation before they can be applied to practical, real life situations. The opposite criticism can be applied to naturalistic research. Critics contend that advocates of field work and case study research sacrifice so much to be able to work in natural settings, that is, to increase the validity of their work, that they make it impossible for anyone else to replicate their findings.
Subjectivity and Objectivity
The word "subjective" is used differently by the two approaches to research, both in reference to what researchers do, and in reference to the data. The experimental-design approach makes every effort to remove the researcher from the research; the goal is to carry out the work as impersonally as possible. The way problems are framed, the methods used, and the manner in which reports are written are all intended to render the work independent of the perspectives, idiosyncrasies, beliefs, or prejudices of the persons carrying it out. The goal is research performed in such a manner that anyone with similar training, given the same situation, could repeat the research protocol, observe the same phenomena, and reach the same conclusions.
Proponents of the naturalistic model argue that, not only is it impossible to remove the researcher from the research, it is better to acknowledge the inevitable presence of the self and capitalize on the researcher's own perspectives and biases. The methods used by naturalistic researchers -- participant observation, clinical interviews, and document analysis -- are precisely those that involve the researcher most directly and personally. The analytic methods favored -- developing categories that emerge from the data, developing taxonomies, and narrative summary descriptions -- require intense cognitive input from the researcher. In addition, the form of writing often used in naturalistic research reports -- relying on quotations and excerpts from the original data -- all emphasize the role of the researcher in the process.
Experimental-design advocates are also skeptical about "subjective" data characteristically collected during naturalistic research. Researchers' own field notes, narrative verbal responses from subjects, or information from individual informants is suspect as unreliable. Melton (1935/1988, p. 7) argued that any efforts to interview visitors would only provide "fallible expressions of [visitors'] interest," rather than the objective data resulting from tracking studies.
In contrast, naturalistic researchers argue that the ability to talk and think about what we have done is one of the great advantages of any research or evaluation activity involving human beings. Using this information can provide insight into understanding the meaning behind people's behavior. It is "subjective," in the sense that it comes from a single subject, but not in the pejorative sense of being particularly unreliable or invalid.
Confirmation and Expected
Outcomes Another distinction between the two approaches involves the outcomes of research. Experimental-design approaches are intended to demonstrate causal connections, to confirm that factor A necessarily causes outcome B. The outcomes from naturalistic inquiry focus much more on understanding and informed description. The intention is not to make a contribution to one overriding, generalized explanatory scheme, but to probe in depth the meaning of the particular situation being studied, as a contribution to a larger, general understanding.
Paradigms and World Hypotheses
The arguments I've briefly sketched above are often characterized as representing different "paradigms" following Kuhn's (1962) groundbreaking examination of the history of science. Kuhn argued that scientific "revolutions" do not represent progress towards truth, but rather the acceptance of new world views, i.e. paradigms. New paradigms replace old ones in science when the new ones succeed in providing satisfactory explanations for particularly striking or important problems, not because they necessarily explain all the facts. Kuhn's formulation has been so popular that the term "paradigm" has now come to stand for almost any difference in perspective on a wide range of subjects.
I would like to suggest that a paradigm shift, as Kuhn developed it, is a very special kind of intellectual change, one that usually implies progression. For example, paradigm shifts are often considered to resemble the kind of change represented by a Piagetian change in intellectual stage. (6) Piagetian stages do represent differences in world view, but they also imply some form of progress; once you accept the conservation of volume you can't go back to the previous stage. Similarly, once you accept the Copernican explanation of planetary motion (to use Kuhn's most powerful example) it is extremely difficult to go back to the Ptolemaic explanation. In fact, there have been no reputable astronomers who advocate the Ptolemaic universe in their research and writing for centuries.
The distinction between acceptance of naturalistic or experimental-design research approaches is quite different from the example above. We have active proponents of both, work goes on side by side (if not always amiably) and proponents of both views abound.
A better model for describing these two world views about research can be found in the classic gestalt psychology metaphor of ambiguous visual images. Look at Figure 10. Do you see a duck or a rabbit?(7) Ambiguous figures best illustrate the distinction between contrasting world-views. They represent truly alternative ways of interpreting the same data; it is impossible to argue that one perception is better, more advanced, or more accurate than the other.
In fact, it is precisely such ambiguous representations that Campbell (1978) uses to argue that experimental design research owes much to its naturalistic underpinnings, and that we need to recognize the value of both approaches. (8) He uses several of the famous ambiguous drawings in developing his argument along lines similar to those presented here.
A formulation of differences between "paradigms" that avoids some of the problems mentioned above can be found in Pepper's (1942) conception of world hypotheses. Pepper's analysis of how knowledge is constructed and how knowledge systems are justified leads him to assert that there are four distinct "world hypotheses" -- ways of looking at the world and our explanations of it -- that cannot be justified by any reference to arguments outside their own systems of thought. Each, like the alternative figures of the ambiguous drawings, is consistent within its own boundaries but does not necessarily meet criteria for acceptance when viewed from the perspective of another world hypothesis. He postulates four viable world hypotheses, formalism, mechanism, organicism, and contextualism, and suggests that each is developed out of a unique root metaphor.(9)
The arguments that I have briefly introduced in the discussion above suggest that we will not be able to resolve the debate between proponents of mazes and webs. (10) We need to acknowledge that they represent distinct ways of looking at the world, each with advantages and disadvantages. Our own field will be strengthened to the extent that we pursue both approaches and acknowledge their unique contributions to our understanding of museum visitors.
Maze and Web World Hypotheses
The world hypothesis that supports a constructivist view of education is the same one that governs the arguments in support of naturalistic research, just as traditional views of learning and teaching share a world view with experimental-design research.
If we believe that the subject to be learned can be structured hierarchically and that a lower order concept needs to be learned before a higher order one can be comprehended, then it makes sense to set up a research or evaluation protocol which will test such acquisition of learning; a system that isolates variables, assesses the learning of individual concepts, and builds a piece by piece understanding of the education that is going on.
In contrast, if we believe that education is impossible to extricate from its context, occurs as a totality resulting in individual meaning making, based on a large number of interacting factors, then we are more likely to develop a research approach to studying learning that embraces and accommodates this view of education, namely, a naturalistic research approach.
In Figure 11, I have outlined some alternative criteria that characterize traditional didactic/experimental-design views versus those that characterize constructivist/ naturalistic views. I look forward to discussing this distinction with you further and debating the merits of each world view, both at this 10th anniversary Visitor Studies Association meeting and in the future.