what is not a visual representation

• Open access
• Published: 19 July 2015

The role of visual representations in scientific practices: from conceptual understanding and knowledge generation to ‘seeing’ how science works

• Maria Evagorou 1 ,
• Sibel Erduran 2 &
• Terhi Mäntylä 3  

International Journal of STEM Education volume  2 , Article number:  11 ( 2015 ) Cite this article

75k Accesses

78 Citations

13 Altmetric

Metrics details

The use of visual representations (i.e., photographs, diagrams, models) has been part of science, and their use makes it possible for scientists to interact with and represent complex phenomena, not observable in other ways. Despite a wealth of research in science education on visual representations, the emphasis of such research has mainly been on the conceptual understanding when using visual representations and less on visual representations as epistemic objects. In this paper, we argue that by positioning visual representations as epistemic objects of scientific practices, science education can bring a renewed focus on how visualization contributes to knowledge formation in science from the learners’ perspective.

This is a theoretical paper, and in order to argue about the role of visualization, we first present a case study, that of the discovery of the structure of DNA that highlights the epistemic components of visual information in science. The second case study focuses on Faraday’s use of the lines of magnetic force. Faraday is known of his exploratory, creative, and yet systemic way of experimenting, and the visual reasoning leading to theoretical development was an inherent part of the experimentation. Third, we trace a contemporary account from science focusing on the experimental practices and how reproducibility of experimental procedures can be reinforced through video data.

Conclusions

Our conclusions suggest that in teaching science, the emphasis in visualization should shift from cognitive understanding—using the products of science to understand the content—to engaging in the processes of visualization. Furthermore, we suggest that is it essential to design curriculum materials and learning environments that create a social and epistemic context and invite students to engage in the practice of visualization as evidence, reasoning, experimental procedure, or a means of communication and reflect on these practices. Implications for teacher education include the need for teacher professional development programs to problematize the use of visual representations as epistemic objects that are part of scientific practices.

During the last decades, research and reform documents in science education across the world have been calling for an emphasis not only on the content but also on the processes of science (Bybee 2014 ; Eurydice 2012 ; Duschl and Bybee 2014 ; Osborne 2014 ; Schwartz et al. 2012 ), in order to make science accessible to the students and enable them to understand the epistemic foundation of science. Scientific practices, part of the process of science, are the cognitive and discursive activities that are targeted in science education to develop epistemic understanding and appreciation of the nature of science (Duschl et al. 2008 ) and have been the emphasis of recent reform documents in science education across the world (Achieve 2013 ; Eurydice 2012 ). With the term scientific practices, we refer to the processes that take place during scientific discoveries and include among others: asking questions, developing and using models, engaging in arguments, and constructing and communicating explanations (National Research Council 2012 ). The emphasis on scientific practices aims to move the teaching of science from knowledge to the understanding of the processes and the epistemic aspects of science. Additionally, by placing an emphasis on engaging students in scientific practices, we aim to help students acquire scientific knowledge in meaningful contexts that resemble the reality of scientific discoveries.

Despite a wealth of research in science education on visual representations, the emphasis of such research has mainly been on the conceptual understanding when using visual representations and less on visual representations as epistemic objects. In this paper, we argue that by positioning visual representations as epistemic objects, science education can bring a renewed focus on how visualization contributes to knowledge formation in science from the learners’ perspective. Specifically, the use of visual representations (i.e., photographs, diagrams, tables, charts) has been part of science and over the years has evolved with the new technologies (i.e., from drawings to advanced digital images and three dimensional models). Visualization makes it possible for scientists to interact with complex phenomena (Richards 2003 ), and they might convey important evidence not observable in other ways. Visual representations as a tool to support cognitive understanding in science have been studied extensively (i.e., Gilbert 2010 ; Wu and Shah 2004 ). Studies in science education have explored the use of images in science textbooks (i.e., Dimopoulos et al. 2003 ; Bungum 2008 ), students’ representations or models when doing science (i.e., Gilbert et al. 2008 ; Dori et al. 2003 ; Lehrer and Schauble 2012 ; Schwarz et al. 2009 ), and students’ images of science and scientists (i.e., Chambers 1983 ). Therefore, studies in the field of science education have been using the term visualization as “the formation of an internal representation from an external representation” (Gilbert et al. 2008 , p. 4) or as a tool for conceptual understanding for students.

In this paper, we do not refer to visualization as mental image, model, or presentation only (Gilbert et al. 2008 ; Philips et al. 2010 ) but instead focus on visual representations or visualization as epistemic objects. Specifically, we refer to visualization as a process for knowledge production and growth in science. In this respect, modeling is an aspect of visualization, but what we are focusing on with visualization is not on the use of model as a tool for cognitive understanding (Gilbert 2010 ; Wu and Shah 2004 ) but the on the process of modeling as a scientific practice which includes the construction and use of models, the use of other representations, the communication in the groups with the use of the visual representation, and the appreciation of the difficulties that the science phase in this process. Therefore, the purpose of this paper is to present through the history of science how visualization can be considered not only as a cognitive tool in science education but also as an epistemic object that can potentially support students to understand aspects of the nature of science.

Scientific practices and science education

According to the New Generation Science Standards (Achieve 2013 ), scientific practices refer to: asking questions and defining problems; developing and using models; planning and carrying out investigations; analyzing and interpreting data; using mathematical and computational thinking; constructing explanations and designing solutions; engaging in argument from evidence; and obtaining, evaluating, and communicating information. A significant aspect of scientific practices is that science learning is more than just about learning facts, concepts, theories, and laws. A fuller appreciation of science necessitates the understanding of the science relative to its epistemological grounding and the process that are involved in the production of knowledge (Hogan and Maglienti 2001 ; Wickman 2004 ).

The New Generation Science Standards is, among other changes, shifting away from science inquiry and towards the inclusion of scientific practices (Duschl and Bybee 2014 ; Osborne 2014 ). By comparing the abilities to do scientific inquiry (National Research Council 2000 ) with the set of scientific practices, it is evident that the latter is about engaging in the processes of doing science and experiencing in that way science in a more authentic way. Engaging in scientific practices according to Osborne ( 2014 ) “presents a more authentic picture of the endeavor that is science” (p.183) and also helps the students to develop a deeper understanding of the epistemic aspects of science. Furthermore, as Bybee ( 2014 ) argues, by engaging students in scientific practices, we involve them in an understanding of the nature of science and an understanding on the nature of scientific knowledge.

Science as a practice and scientific practices as a term emerged by the philosopher of science, Kuhn (Osborne 2014 ), refers to the processes in which the scientists engage during knowledge production and communication. The work that is followed by historians, philosophers, and sociologists of science (Latour 2011 ; Longino 2002 ; Nersessian 2008 ) revealed the scientific practices in which the scientists engage in and include among others theory development and specific ways of talking, modeling, and communicating the outcomes of science.

Visualization as an epistemic object

Schematic, pictorial symbols in the design of scientific instruments and analysis of the perceptual and functional information that is being stored in those images have been areas of investigation in philosophy of scientific experimentation (Gooding et al. 1993 ). The nature of visual perception, the relationship between thought and vision, and the role of reproducibility as a norm for experimental research form a central aspect of this domain of research in philosophy of science. For instance, Rothbart ( 1997 ) has argued that visualizations are commonplace in the theoretical sciences even if every scientific theory may not be defined by visualized models.

Visual representations (i.e., photographs, diagrams, tables, charts, models) have been used in science over the years to enable scientists to interact with complex phenomena (Richards 2003 ) and might convey important evidence not observable in other ways (Barber et al. 2006 ). Some authors (e.g., Ruivenkamp and Rip 2010 ) have argued that visualization is as a core activity of some scientific communities of practice (e.g., nanotechnology) while others (e.g., Lynch and Edgerton 1988 ) have differentiated the role of particular visualization techniques (e.g., of digital image processing in astronomy). Visualization in science includes the complex process through which scientists develop or produce imagery, schemes, and graphical representation, and therefore, what is of importance in this process is not only the result but also the methodology employed by the scientists, namely, how this result was produced. Visual representations in science may refer to objects that are believed to have some kind of material or physical existence but equally might refer to purely mental, conceptual, and abstract constructs (Pauwels 2006 ). More specifically, visual representations can be found for: (a) phenomena that are not observable with the eye (i.e., microscopic or macroscopic); (b) phenomena that do not exist as visual representations but can be translated as such (i.e., sound); and (c) in experimental settings to provide visual data representations (i.e., graphs presenting velocity of moving objects). Additionally, since science is not only about replicating reality but also about making it more understandable to people (either to the public or other scientists), visual representations are not only about reproducing the nature but also about: (a) functioning in helping solving a problem, (b) filling gaps in our knowledge, and (c) facilitating knowledge building or transfer (Lynch 2006 ).

Using or developing visual representations in the scientific practice can range from a straightforward to a complicated situation. More specifically, scientists can observe a phenomenon (i.e., mitosis) and represent it visually using a picture or diagram, which is quite straightforward. But they can also use a variety of complicated techniques (i.e., crystallography in the case of DNA studies) that are either available or need to be developed or refined in order to acquire the visual information that can be used in the process of theory development (i.e., Latour and Woolgar 1979 ). Furthermore, some visual representations need decoding, and the scientists need to learn how to read these images (i.e., radiologists); therefore, using visual representations in the process of science requires learning a new language that is specific to the medium/methods that is used (i.e., understanding an X-ray picture is different from understanding an MRI scan) and then communicating that language to other scientists and the public.

There are much intent and purposes of visual representations in scientific practices, as for example to make a diagnosis, compare, describe, and preserve for future study, verify and explore new territory, generate new data (Pauwels 2006 ), or present new methodologies. According to Latour and Woolgar ( 1979 ) and Knorr Cetina ( 1999 ), visual representations can be used either as primary data (i.e., image from a microscope). or can be used to help in concept development (i.e., models of DNA used by Watson and Crick), to uncover relationships and to make the abstract more concrete (graphs of sound waves). Therefore, visual representations and visual practices, in all forms, are an important aspect of the scientific practices in developing, clarifying, and transmitting scientific knowledge (Pauwels 2006 ).

Methods and Results: Merging Visualization and scientific practices in science

In this paper, we present three case studies that embody the working practices of scientists in an effort to present visualization as a scientific practice and present our argument about how visualization is a complex process that could include among others modeling and use of representation but is not only limited to that. The first case study explores the role of visualization in the construction of knowledge about the structure of DNA, using visuals as evidence. The second case study focuses on Faraday’s use of the lines of magnetic force and the visual reasoning leading to the theoretical development that was an inherent part of the experimentation. The third case study focuses on the current practices of scientists in the context of a peer-reviewed journal called the Journal of Visualized Experiments where the methodology is communicated through videotaped procedures. The three case studies represent the research interests of the three authors of this paper and were chosen to present how visualization as a practice can be involved in all stages of doing science, from hypothesizing and evaluating evidence (case study 1) to experimenting and reasoning (case study 2) to communicating the findings and methodology with the research community (case study 3), and represent in this way the three functions of visualization as presented by Lynch ( 2006 ). Furthermore, the last case study showcases how the development of visualization technologies has contributed to the communication of findings and methodologies in science and present in that way an aspect of current scientific practices. In all three cases, our approach is guided by the observation that the visual information is an integral part of scientific practices at the least and furthermore that they are particularly central in the scientific practices of science.

Case study 1: use visual representations as evidence in the discovery of DNA

The focus of the first case study is the discovery of the structure of DNA. The DNA was first isolated in 1869 by Friedrich Miescher, and by the late 1940s, it was known that it contained phosphate, sugar, and four nitrogen-containing chemical bases. However, no one had figured the structure of the DNA until Watson and Crick presented their model of DNA in 1953. Other than the social aspects of the discovery of the DNA, another important aspect was the role of visual evidence that led to knowledge development in the area. More specifically, by studying the personal accounts of Watson ( 1968 ) and Crick ( 1988 ) about the discovery of the structure of the DNA, the following main ideas regarding the role of visual representations in the production of knowledge can be identified: (a) The use of visual representations was an important part of knowledge growth and was often dependent upon the discovery of new technologies (i.e., better microscopes or better techniques in crystallography that would provide better visual representations as evidence of the helical structure of the DNA); and (b) Models (three-dimensional) were used as a way to represent the visual images (X-ray images) and connect them to the evidence provided by other sources to see whether the theory can be supported. Therefore, the model of DNA was built based on the combination of visual evidence and experimental data.

An example showcasing the importance of visual representations in the process of knowledge production in this case is provided by Watson, in his book The Double Helix (1968):

…since the middle of the summer Rosy [Rosalind Franklin] had had evidence for a new three-dimensional form of DNA. It occurred when the DNA 2molecules were surrounded by a large amount of water. When I asked what the pattern was like, Maurice went into the adjacent room to pick up a print of the new form they called the “B” structure. The instant I saw the picture, my mouth fell open and my pulse began to race. The pattern was unbelievably simpler than those previously obtained (A form). Moreover, the black cross of reflections which dominated the picture could arise only from a helical structure. With the A form the argument for the helix was never straightforward, and considerable ambiguity existed as to exactly which type of helical symmetry was present. With the B form however, mere inspection of its X-ray picture gave several of the vital helical parameters. (p. 167-169)

As suggested by Watson’s personal account of the discovery of the DNA, the photo taken by Rosalind Franklin (Fig.  1 ) convinced him that the DNA molecule must consist of two chains arranged in a paired helix, which resembles a spiral staircase or ladder, and on March 7, 1953, Watson and Crick finished and presented their model of the structure of DNA (Watson and Berry 2004 ; Watson 1968 ) which was based on the visual information provided by the X-ray image and their knowledge of chemistry.

X-ray chrystallography of DNA

In analyzing the visualization practice in this case study, we observe the following instances that highlight how the visual information played a role:

Asking questions and defining problems: The real world in the model of science can at some points only be observed through visual representations or representations, i.e., if we are using DNA as an example, the structure of DNA was only observable through the crystallography images produced by Rosalind Franklin in the laboratory. There was no other way to observe the structure of DNA, therefore the real world.

Analyzing and interpreting data: The images that resulted from crystallography as well as their interpretations served as the data for the scientists studying the structure of DNA.

Experimenting: The data in the form of visual information were used to predict the possible structure of the DNA.

Modeling: Based on the prediction, an actual three-dimensional model was prepared by Watson and Crick. The first model did not fit with the real world (refuted by Rosalind Franklin and her research group from King’s College) and Watson and Crick had to go through the same process again to find better visual evidence (better crystallography images) and create an improved visual model.

Example excerpts from Watson’s biography provide further evidence for how visualization practices were applied in the context of the discovery of DNA (Table  1 ).

In summary, by examining the history of the discovery of DNA, we showcased how visual data is used as scientific evidence in science, identifying in that way an aspect of the nature of science that is still unexplored in the history of science and an aspect that has been ignored in the teaching of science. Visual representations are used in many ways: as images, as models, as evidence to support or rebut a model, and as interpretations of reality.

Case study 2: applying visual reasoning in knowledge production, the example of the lines of magnetic force

The focus of this case study is on Faraday’s use of the lines of magnetic force. Faraday is known of his exploratory, creative, and yet systemic way of experimenting, and the visual reasoning leading to theoretical development was an inherent part of this experimentation (Gooding 2006 ). Faraday’s articles or notebooks do not include mathematical formulations; instead, they include images and illustrations from experimental devices and setups to the recapping of his theoretical ideas (Nersessian 2008 ). According to Gooding ( 2006 ), “Faraday’s visual method was designed not to copy apparent features of the world, but to analyse and replicate them” (2006, p. 46).

The lines of force played a central role in Faraday’s research on electricity and magnetism and in the development of his “field theory” (Faraday 1852a ; Nersessian 1984 ). Before Faraday, the experiments with iron filings around magnets were known and the term “magnetic curves” was used for the iron filing patterns and also for the geometrical constructs derived from the mathematical theory of magnetism (Gooding et al. 1993 ). However, Faraday used the lines of force for explaining his experimental observations and in constructing the theory of forces in magnetism and electricity. Examples of Faraday’s different illustrations of lines of magnetic force are given in Fig.  2 . Faraday gave the following experiment-based definition for the lines of magnetic forces:

a Iron filing pattern in case of bar magnet drawn by Faraday (Faraday 1852b , Plate IX, p. 158, Fig. 1), b Faraday’s drawing of lines of magnetic force in case of cylinder magnet, where the experimental procedure, knife blade showing the direction of lines, is combined into drawing (Faraday, 1855, vol. 1, plate 1)

A line of magnetic force may be defined as that line which is described by a very small magnetic needle, when it is so moved in either direction correspondent to its length, that the needle is constantly a tangent to the line of motion; or it is that line along which, if a transverse wire be moved in either direction, there is no tendency to the formation of any current in the wire, whilst if moved in any other direction there is such a tendency; or it is that line which coincides with the direction of the magnecrystallic axis of a crystal of bismuth, which is carried in either direction along it. The direction of these lines about and amongst magnets and electric currents, is easily represented and understood, in a general manner, by the ordinary use of iron filings. (Faraday 1852a , p. 25 (3071))

The definition describes the connection between the experiments and the visual representation of the results. Initially, the lines of force were just geometric representations, but later, Faraday treated them as physical objects (Nersessian 1984 ; Pocovi and Finlay 2002 ):

I have sometimes used the term lines of force so vaguely, as to leave the reader doubtful whether I intended it as a merely representative idea of the forces, or as the description of the path along which the power was continuously exerted. … wherever the expression line of force is taken simply to represent the disposition of forces, it shall have the fullness of that meaning; but that wherever it may seem to represent the idea of the physical mode of transmission of the force, it expresses in that respect the opinion to which I incline at present. The opinion may be erroneous, and yet all that relates or refers to the disposition of the force will remain the same. (Faraday, 1852a , p. 55-56 (3075))

He also felt that the lines of force had greater explanatory power than the dominant theory of action-at-a-distance:

Now it appears to me that these lines may be employed with great advantage to represent nature, condition, direction and comparative amount of the magnetic forces; and that in many cases they have, to the physical reasoned at least, a superiority over that method which represents the forces as concentrated in centres of action… (Faraday, 1852a , p. 26 (3074))

For giving some insight to Faraday’s visual reasoning as an epistemic practice, the following examples of Faraday’s studies of the lines of magnetic force (Faraday 1852a , 1852b ) are presented:

(a) Asking questions and defining problems: The iron filing patterns formed the empirical basis for the visual model: 2D visualization of lines of magnetic force as presented in Fig.  2 . According to Faraday, these iron filing patterns were suitable for illustrating the direction and form of the magnetic lines of force (emphasis added):

It must be well understood that these forms give no indication by their appearance of the relative strength of the magnetic force at different places, inasmuch as the appearance of the lines depends greatly upon the quantity of filings and the amount of tapping; but the direction and forms of these lines are well given, and these indicate, in a considerable degree, the direction in which the forces increase and diminish . (Faraday 1852b , p.158 (3237))

Despite being static and two dimensional on paper, the lines of magnetic force were dynamical (Nersessian 1992 , 2008 ) and three dimensional for Faraday (see Fig.  2 b). For instance, Faraday described the lines of force “expanding”, “bending,” and “being cut” (Nersessian 1992 ). In Fig.  2 b, Faraday has summarized his experiment (bar magnet and knife blade) and its results (lines of force) in one picture.

(b) Analyzing and interpreting data: The model was so powerful for Faraday that he ended up thinking them as physical objects (e.g., Nersessian 1984 ), i.e., making interpretations of the way forces act. Of course, he made a lot of experiments for showing the physical existence of the lines of force, but he did not succeed in it (Nersessian 1984 ). The following quote illuminates Faraday’s use of the lines of force in different situations:

The study of these lines has, at different times, been greatly influential in leading me to various results, which I think prove their utility as well as fertility. Thus, the law of magneto-electric induction; the earth’s inductive action; the relation of magnetism and light; diamagnetic action and its law, and magnetocrystallic action, are the cases of this kind… (Faraday 1852a , p. 55 (3174))

(c) Experimenting: In Faraday's case, he used a lot of exploratory experiments; in case of lines of magnetic force, he used, e.g., iron filings, magnetic needles, or current carrying wires (see the quote above). The magnetic field is not directly observable and the representation of lines of force was a visual model, which includes the direction, form, and magnitude of field.

(d) Modeling: There is no denying that the lines of magnetic force are visual by nature. Faraday’s views of lines of force developed gradually during the years, and he applied and developed them in different contexts such as electromagnetic, electrostatic, and magnetic induction (Nersessian 1984 ). An example of Faraday’s explanation of the effect of the wire b’s position to experiment is given in Fig.  3 . In Fig.  3 , few magnetic lines of force are drawn, and in the quote below, Faraday is explaining the effect using these magnetic lines of force (emphasis added):

Picture of an experiment with different arrangements of wires ( a , b’ , b” ), magnet, and galvanometer. Note the lines of force drawn around the magnet. (Faraday 1852a , p. 34)

It will be evident by inspection of Fig. 3 , that, however the wires are carried away, the general result will, according to the assumed principles of action, be the same; for if a be the axial wire, and b’, b”, b”’ the equatorial wire, represented in three different positions, whatever magnetic lines of force pass across the latter wire in one position, will also pass it in the other, or in any other position which can be given to it. The distance of the wire at the place of intersection with the lines of force, has been shown, by the experiments (3093.), to be unimportant. (Faraday 1852a , p. 34 (3099))

In summary, by examining the history of Faraday’s use of lines of force, we showed how visual imagery and reasoning played an important part in Faraday’s construction and representation of his “field theory”. As Gooding has stated, “many of Faraday’s sketches are far more that depictions of observation, they are tools for reasoning with and about phenomena” (2006, p. 59).

Case study 3: visualizing scientific methods, the case of a journal

The focus of the third case study is the Journal of Visualized Experiments (JoVE) , a peer-reviewed publication indexed in PubMed. The journal devoted to the publication of biological, medical, chemical, and physical research in a video format. The journal describes its history as follows:

JoVE was established as a new tool in life science publication and communication, with participation of scientists from leading research institutions. JoVE takes advantage of video technology to capture and transmit the multiple facets and intricacies of life science research. Visualization greatly facilitates the understanding and efficient reproduction of both basic and complex experimental techniques, thereby addressing two of the biggest challenges faced by today's life science research community: i) low transparency and poor reproducibility of biological experiments and ii) time and labor-intensive nature of learning new experimental techniques. ( http://www.jove.com/ )

By examining the journal content, we generate a set of categories that can be considered as indicators of relevance and significance in terms of epistemic practices of science that have relevance for science education. For example, the quote above illustrates how scientists view some norms of scientific practice including the norms of “transparency” and “reproducibility” of experimental methods and results, and how the visual format of the journal facilitates the implementation of these norms. “Reproducibility” can be considered as an epistemic criterion that sits at the heart of what counts as an experimental procedure in science:

Investigating what should be reproducible and by whom leads to different types of experimental reproducibility, which can be observed to play different roles in experimental practice. A successful application of the strategy of reproducing an experiment is an achievement that may depend on certain isiosyncratic aspects of a local situation. Yet a purely local experiment that cannot be carried out by other experimenters and in other experimental contexts will, in the end be unproductive in science. (Sarkar and Pfeifer 2006 , p.270)

We now turn to an article on “Elevated Plus Maze for Mice” that is available for free on the journal website ( http://www.jove.com/video/1088/elevated-plus-maze-for-mice ). The purpose of this experiment was to investigate anxiety levels in mice through behavioral analysis. The journal article consists of a 9-min video accompanied by text. The video illustrates the handling of the mice in soundproof location with less light, worksheets with characteristics of mice, computer software, apparatus, resources, setting up the computer software, and the video recording of mouse behavior on the computer. The authors describe the apparatus that is used in the experiment and state how procedural differences exist between research groups that lead to difficulties in the interpretation of results:

The apparatus consists of open arms and closed arms, crossed in the middle perpendicularly to each other, and a center area. Mice are given access to all of the arms and are allowed to move freely between them. The number of entries into the open arms and the time spent in the open arms are used as indices of open space-induced anxiety in mice. Unfortunately, the procedural differences that exist between laboratories make it difficult to duplicate and compare results among laboratories.

The authors’ emphasis on the particularity of procedural context echoes in the observations of some philosophers of science:

It is not just the knowledge of experimental objects and phenomena but also their actual existence and occurrence that prove to be dependent on specific, productive interventions by the experimenters” (Sarkar and Pfeifer 2006 , pp. 270-271)

The inclusion of a video of the experimental procedure specifies what the apparatus looks like (Fig.  4 ) and how the behavior of the mice is captured through video recording that feeds into a computer (Fig.  5 ). Subsequently, a computer software which captures different variables such as the distance traveled, the number of entries, and the time spent on each arm of the apparatus. Here, there is visual information at different levels of representation ranging from reconfiguration of raw video data to representations that analyze the data around the variables in question (Fig.  6 ). The practice of levels of visual representations is not particular to the biological sciences. For instance, they are commonplace in nanotechnological practices:

Visual illustration of apparatus

Video processing of experimental set-up

Computer software for video input and variable recording

In the visualization processes, instruments are needed that can register the nanoscale and provide raw data, which needs to be transformed into images. Some Imaging Techniques have software incorporated already where this transformation automatically takes place, providing raw images. Raw data must be translated through the use of Graphic Software and software is also used for the further manipulation of images to highlight what is of interest to capture the (inferred) phenomena -- and to capture the reader. There are two levels of choice: Scientists have to choose which imaging technique and embedded software to use for the job at hand, and they will then have to follow the structure of the software. Within such software, there are explicit choices for the scientists, e.g. about colour coding, and ways of sharpening images. (Ruivenkamp and Rip 2010 , pp.14–15)

On the text that accompanies the video, the authors highlight the role of visualization in their experiment:

Visualization of the protocol will promote better understanding of the details of the entire experimental procedure, allowing for standardization of the protocols used in different laboratories and comparisons of the behavioral phenotypes of various strains of mutant mice assessed using this test.

The software that takes the video data and transforms it into various representations allows the researchers to collect data on mouse behavior more reliably. For instance, the distance traveled across the arms of the apparatus or the time spent on each arm would have been difficult to observe and record precisely. A further aspect to note is how the visualization of the experiment facilitates control of bias. The authors illustrate how the olfactory bias between experimental procedures carried on mice in sequence is avoided by cleaning the equipment.

Our discussion highlights the role of visualization in science, particularly with respect to presenting visualization as part of the scientific practices. We have used case studies from the history of science highlighting a scientist’s account of how visualization played a role in the discovery of DNA and the magnetic field and from a contemporary illustration of a science journal’s practices in incorporating visualization as a way to communicate new findings and methodologies. Our implicit aim in drawing from these case studies was the need to align science education with scientific practices, particularly in terms of how visual representations, stable or dynamic, can engage students in the processes of science and not only to be used as tools for cognitive development in science. Our approach was guided by the notion of “knowledge-as-practice” as advanced by Knorr Cetina ( 1999 ) who studied scientists and characterized their knowledge as practice, a characterization which shifts focus away from ideas inside scientists’ minds to practices that are cultural and deeply contextualized within fields of science. She suggests that people working together can be examined as epistemic cultures whose collective knowledge exists as practice.

It is important to stress, however, that visual representations are not used in isolation, but are supported by other types of evidence as well, or other theories (i.e., in order to understand the helical form of DNA, or the structure, chemistry knowledge was needed). More importantly, this finding can also have implications when teaching science as argument (e.g., Erduran and Jimenez-Aleixandre 2008 ), since the verbal evidence used in the science classroom to maintain an argument could be supported by visual evidence (either a model, representation, image, graph, etc.). For example, in a group of students discussing the outcomes of an introduced species in an ecosystem, pictures of the species and the ecosystem over time, and videos showing the changes in the ecosystem, and the special characteristics of the different species could serve as visual evidence to help the students support their arguments (Evagorou et al. 2012 ). Therefore, an important implication for the teaching of science is the use of visual representations as evidence in the science curriculum as part of knowledge production. Even though studies in the area of science education have focused on the use of models and modeling as a way to support students in the learning of science (Dori et al. 2003 ; Lehrer and Schauble 2012 ; Mendonça and Justi 2013 ; Papaevripidou et al. 2007 ) or on the use of images (i.e., Korfiatis et al. 2003 ), with the term using visuals as evidence, we refer to the collection of all forms of visuals and the processes involved.

Another aspect that was identified through the case studies is that of the visual reasoning (an integral part of Faraday’s investigations). Both the verbalization and visualization were part of the process of generating new knowledge (Gooding 2006 ). Even today, most of the textbooks use the lines of force (or just field lines) as a geometrical representation of field, and the number of field lines is connected to the quantity of flux. Often, the textbooks use the same kind of visual imagery than in what is used by scientists. However, when using images, only certain aspects or features of the phenomena or data are captured or highlighted, and often in tacit ways. Especially in textbooks, the process of producing the image is not presented and instead only the product—image—is left. This could easily lead to an idea of images (i.e., photos, graphs, visual model) being just representations of knowledge and, in the worse case, misinterpreted representations of knowledge as the results of Pocovi and Finlay ( 2002 ) in case of electric field lines show. In order to avoid this, the teachers should be able to explain how the images are produced (what features of phenomena or data the images captures, on what ground the features are chosen to that image, and what features are omitted); in this way, the role of visualization in knowledge production can be made “visible” to students by engaging them in the process of visualization.

The implication of these norms for science teaching and learning is numerous. The classroom contexts can model the generation, sharing and evaluation of evidence, and experimental procedures carried out by students, thereby promoting not only some contemporary cultural norms in scientific practice but also enabling the learning of criteria, standards, and heuristics that scientists use in making decisions on scientific methods. As we have demonstrated with the three case studies, visual representations are part of the process of knowledge growth and communication in science, as demonstrated with two examples from the history of science and an example from current scientific practices. Additionally, visual information, especially with the use of technology is a part of students’ everyday lives. Therefore, we suggest making use of students’ knowledge and technological skills (i.e., how to produce their own videos showing their experimental method or how to identify or provide appropriate visual evidence for a given topic), in order to teach them the aspects of the nature of science that are often neglected both in the history of science and the design of curriculum. Specifically, what we suggest in this paper is that students should actively engage in visualization processes in order to appreciate the diverse nature of doing science and engage in authentic scientific practices.

However, as a word of caution, we need to distinguish the products and processes involved in visualization practices in science:

If one considers scientific representations and the ways in which they can foster or thwart our understanding, it is clear that a mere object approach, which would devote all attention to the representation as a free-standing product of scientific labor, is inadequate. What is needed is a process approach: each visual representation should be linked with its context of production (Pauwels 2006 , p.21).

The aforementioned suggests that the emphasis in visualization should shift from cognitive understanding—using the products of science to understand the content—to engaging in the processes of visualization. Therefore, an implication for the teaching of science includes designing curriculum materials and learning environments that create a social and epistemic context and invite students to engage in the practice of visualization as evidence, reasoning, experimental procedure, or a means of communication (as presented in the three case studies) and reflect on these practices (Ryu et al. 2015 ).

Finally, a question that arises from including visualization in science education, as well as from including scientific practices in science education is whether teachers themselves are prepared to include them as part of their teaching (Bybee 2014 ). Teacher preparation programs and teacher education have been critiqued, studied, and rethought since the time they emerged (Cochran-Smith 2004 ). Despite the years of history in teacher training and teacher education, the debate about initial teacher training and its content still pertains in our community and in policy circles (Cochran-Smith 2004 ; Conway et al. 2009 ). In the last decades, the debate has shifted from a behavioral view of learning and teaching to a learning problem—focusing on that way not only on teachers’ knowledge, skills, and beliefs but also on making the connection of the aforementioned with how and if pupils learn (Cochran-Smith 2004 ). The Science Education in Europe report recommended that “Good quality teachers, with up-to-date knowledge and skills, are the foundation of any system of formal science education” (Osborne and Dillon 2008 , p.9).

However, questions such as what should be the emphasis on pre-service and in-service science teacher training, especially with the new emphasis on scientific practices, still remain unanswered. As Bybee ( 2014 ) argues, starting from the new emphasis on scientific practices in the NGSS, we should consider teacher preparation programs “that would provide undergraduates opportunities to learn the science content and practices in contexts that would be aligned with their future work as teachers” (p.218). Therefore, engaging pre- and in-service teachers in visualization as a scientific practice should be one of the purposes of teacher preparation programs.

Achieve. (2013). The next generation science standards (pp. 1–3). Retrieved from http://www.nextgenscience.org/ .

Google Scholar  

Barber, J, Pearson, D, & Cervetti, G. (2006). Seeds of science/roots of reading . California: The Regents of the University of California.

Bungum, B. (2008). Images of physics: an explorative study of the changing character of visual images in Norwegian physics textbooks. NorDiNa, 4 (2), 132–141.

Bybee, RW. (2014). NGSS and the next generation of science teachers. Journal of Science Teacher Education, 25 (2), 211–221. doi: 10.1007/s10972-014-9381-4 .

Article   Google Scholar  

Chambers, D. (1983). Stereotypic images of the scientist: the draw-a-scientist test. Science Education, 67 (2), 255–265.

Cochran-Smith, M. (2004). The problem of teacher education. Journal of Teacher Education, 55 (4), 295–299. doi: 10.1177/0022487104268057 .

Conway, PF, Murphy, R, & Rath, A. (2009). Learning to teach and its implications for the continuum of teacher education: a nine-country cross-national study .

Crick, F. (1988). What a mad pursuit . USA: Basic Books.

Dimopoulos, K, Koulaidis, V, & Sklaveniti, S. (2003). Towards an analysis of visual images in school science textbooks and press articles about science and technology. Research in Science Education, 33 , 189–216.

Dori, YJ, Tal, RT, & Tsaushu, M. (2003). Teaching biotechnology through case studies—can we improve higher order thinking skills of nonscience majors? Science Education, 87 (6), 767–793. doi: 10.1002/sce.10081 .

Duschl, RA, & Bybee, RW. (2014). Planning and carrying out investigations: an entry to learning and to teacher professional development around NGSS science and engineering practices. International Journal of STEM Education, 1 (1), 12. doi: 10.1186/s40594-014-0012-6 .

Duschl, R., Schweingruber, H. A., & Shouse, A. (2008). Taking science to school . Washington DC: National Academies Press.

Erduran, S, & Jimenez-Aleixandre, MP (Eds.). (2008). Argumentation in science education: perspectives from classroom-based research . Dordrecht: Springer.

Eurydice. (2012). Developing key competencies at school in Europe: challenges and opportunities for policy – 2011/12 (pp. 1–72).

Evagorou, M, Jimenez-Aleixandre, MP, & Osborne, J. (2012). “Should we kill the grey squirrels?” A study exploring students’ justifications and decision-making. International Journal of Science Education, 34 (3), 401–428. doi: 10.1080/09500693.2011.619211 .

Faraday, M. (1852a). Experimental researches in electricity. – Twenty-eighth series. Philosophical Transactions of the Royal Society of London, 142 , 25–56.

Faraday, M. (1852b). Experimental researches in electricity. – Twenty-ninth series. Philosophical Transactions of the Royal Society of London, 142 , 137–159.

Gilbert, JK. (2010). The role of visual representations in the learning and teaching of science: an introduction (pp. 1–19).

Gilbert, J., Reiner, M. & Nakhleh, M. (2008). Visualization: theory and practice in science education . Dordrecht, The Netherlands: Springer.

Gooding, D. (2006). From phenomenology to field theory: Faraday’s visual reasoning. Perspectives on Science, 14 (1), 40–65.

Gooding, D, Pinch, T, & Schaffer, S (Eds.). (1993). The uses of experiment: studies in the natural sciences . Cambridge: Cambridge University Press.

Hogan, K, & Maglienti, M. (2001). Comparing the epistemological underpinnings of students’ and scientists’ reasoning about conclusions. Journal of Research in Science Teaching, 38 (6), 663–687.

Knorr Cetina, K. (1999). Epistemic cultures: how the sciences make knowledge . Cambridge: Harvard University Press.

Korfiatis, KJ, Stamou, AG, & Paraskevopoulos, S. (2003). Images of nature in Greek primary school textbooks. Science Education, 88 (1), 72–89. doi: 10.1002/sce.10133 .

Latour, B. (2011). Visualisation and cognition: drawing things together (pp. 1–32).

Latour, B, & Woolgar, S. (1979). Laboratory life: the construction of scientific facts . Princeton: Princeton University Press.

Lehrer, R, & Schauble, L. (2012). Seeding evolutionary thinking by engaging children in modeling its foundations. Science Education, 96 (4), 701–724. doi: 10.1002/sce.20475 .

Longino, H. E. (2002). The fate of knowledge . Princeton: Princeton University Press.

Lynch, M. (2006). The production of scientific images: vision and re-vision in the history, philosophy, and sociology of science. In L Pauwels (Ed.), Visual cultures of science: rethinking representational practices in knowledge building and science communication (pp. 26–40). Lebanon, NH: Darthmouth College Press.

Lynch, M. & S. Y. Edgerton Jr. (1988). ‘Aesthetic and digital image processing representational craft in contemporary astronomy’, in G. Fyfe & J. Law (eds), Picturing Power; Visual Depictions and Social Relations (London, Routledge): 184 – 220.

Mendonça, PCC, & Justi, R. (2013). An instrument for analyzing arguments produced in modeling-based chemistry lessons. Journal of Research in Science Teaching, 51 (2), 192–218. doi: 10.1002/tea.21133 .

National Research Council (2000). Inquiry and the national science education standards . Washington DC: National Academies Press.

National Research Council (2012). A framework for K-12 science education . Washington DC: National Academies Press.

Nersessian, NJ. (1984). Faraday to Einstein: constructing meaning in scientific theories . Dordrecht: Martinus Nijhoff Publishers.

Book   Google Scholar  

Nersessian, NJ. (1992). How do scientists think? Capturing the dynamics of conceptual change in science. In RN Giere (Ed.), Cognitive Models of Science (pp. 3–45). Minneapolis: University of Minnesota Press.

Nersessian, NJ. (2008). Creating scientific concepts . Cambridge: The MIT Press.

Osborne, J. (2014). Teaching scientific practices: meeting the challenge of change. Journal of Science Teacher Education, 25 (2), 177–196. doi: 10.1007/s10972-014-9384-1 .

Osborne, J. & Dillon, J. (2008). Science education in Europe: critical reflections . London: Nuffield Foundation.

Papaevripidou, M, Constantinou, CP, & Zacharia, ZC. (2007). Modeling complex marine ecosystems: an investigation of two teaching approaches with fifth graders. Journal of Computer Assisted Learning, 23 (2), 145–157. doi: 10.1111/j.1365-2729.2006.00217.x .

Pauwels, L. (2006). A theoretical framework for assessing visual representational practices in knowledge building and science communications. In L Pauwels (Ed.), Visual cultures of science: rethinking representational practices in knowledge building and science communication (pp. 1–25). Lebanon, NH: Darthmouth College Press.

Philips, L., Norris, S. & McNab, J. (2010). Visualization in mathematics, reading and science education . Dordrecht, The Netherlands: Springer.

Pocovi, MC, & Finlay, F. (2002). Lines of force: Faraday’s and students’ views. Science & Education, 11 , 459–474.

Richards, A. (2003). Argument and authority in the visual representations of science. Technical Communication Quarterly, 12 (2), 183–206. doi: 10.1207/s15427625tcq1202_3 .

Rothbart, D. (1997). Explaining the growth of scientific knowledge: metaphors, models and meaning . Lewiston, NY: Mellen Press.

Ruivenkamp, M, & Rip, A. (2010). Visualizing the invisible nanoscale study: visualization practices in nanotechnology community of practice. Science Studies, 23 (1), 3–36.

Ryu, S, Han, Y, & Paik, S-H. (2015). Understanding co-development of conceptual and epistemic understanding through modeling practices with mobile internet. Journal of Science Education and Technology, 24 (2-3), 330–355. doi: 10.1007/s10956-014-9545-1 .

Sarkar, S, & Pfeifer, J. (2006). The philosophy of science, chapter on experimentation (Vol. 1, A-M). New York: Taylor & Francis.

Schwartz, RS, Lederman, NG, & Abd-el-Khalick, F. (2012). A series of misrepresentations: a response to Allchin’s whole approach to assessing nature of science understandings. Science Education, 96 (4), 685–692. doi: 10.1002/sce.21013 .

Schwarz, CV, Reiser, BJ, Davis, EA, Kenyon, L, Achér, A, Fortus, D, et al. (2009). Developing a learning progression for scientific modeling: making scientific modeling accessible and meaningful for learners. Journal of Research in Science Teaching, 46 (6), 632–654. doi: 10.1002/tea.20311 .

Watson, J. (1968). The Double Helix: a personal account of the discovery of the structure of DNA . New York: Scribner.

Watson, J, & Berry, A. (2004). DNA: the secret of life . New York: Alfred A. Knopf.

Wickman, PO. (2004). The practical epistemologies of the classroom: a study of laboratory work. Science Education, 88 , 325–344.

Wu, HK, & Shah, P. (2004). Exploring visuospatial thinking in chemistry learning. Science Education, 88 (3), 465–492. doi: 10.1002/sce.10126 .

Download references

Acknowledgements

The authors would like to acknowledge all reviewers for their valuable comments that have helped us improve the manuscript.

Author information

Authors and affiliations.

University of Nicosia, 46, Makedonitissa Avenue, Egkomi, 1700, Nicosia, Cyprus

Maria Evagorou

University of Limerick, Limerick, Ireland

Sibel Erduran

University of Tampere, Tampere, Finland

Terhi Mäntylä

You can also search for this author in PubMed   Google Scholar

Corresponding author

Correspondence to Maria Evagorou .

Additional information

Competing interests.

The authors declare that they have no competing interests.

Authors’ contributions

ME carried out the introductory literature review, the analysis of the first case study, and drafted the manuscript. SE carried out the analysis of the third case study and contributed towards the “Conclusions” section of the manuscript. TM carried out the second case study. All authors read and approved the final manuscript.

Rights and permissions

Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License ( https://creativecommons.org/licenses/by/4.0 ), which permits use, duplication, adaptation, distribution, and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.

Reprints and permissions

About this article

Cite this article.

Evagorou, M., Erduran, S. & Mäntylä, T. The role of visual representations in scientific practices: from conceptual understanding and knowledge generation to ‘seeing’ how science works. IJ STEM Ed 2 , 11 (2015). https://doi.org/10.1186/s40594-015-0024-x

Download citation

Received : 29 September 2014

Accepted : 16 May 2015

Published : 19 July 2015

DOI : https://doi.org/10.1186/s40594-015-0024-x

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Visual representations
• Epistemic practices
• Science learning

Change Password

Your password must have 8 characters or more and contain 3 of the following:.

• a lower case character, 
• an upper case character, 
• a special character 

Password Changed Successfully

Your password has been changed

• Sign in / Register

Request Username

Can't sign in? Forgot your username?

Enter your email address below and we will send you your username

If the address matches an existing account you will receive an email with instructions to retrieve your username

Teaching and Learning Science through Multiple Representations: Intuitions and Executive Functions

• Janice Hansen
• Lindsey Engle Richland

Director of Undergraduate Education, School of Education, University of California Irvine, Irvine, CA 92697

Search for more papers by this author

*Address correspondence to: Lindsey Richland ( E-mail Address: [email protected] ; uciscienceoflearning.org).

Associate Professor of Education, School of Education, University of California Irvine, Irvine, CA 92697

Reasoning about visual representations in science requires the ability to control one’s attention, inhibit attention to irrelevant or incorrect information, and hold information in mind while manipulating it actively—all aspects of the limited-capacity cognitive system described as humans’ executive functions. This article describes pedagogical intuitions on best practices for how to sequence visual representations among pre-service teachers, adult undergraduates, and middle school children, with learning also tested in the middle school sample. Interestingly, at all ages, most people reported beliefs about teaching others that were different from beliefs about how they would learn. Teaching beliefs were most often that others would learn better from presenting representations one at a time, serially; while learning beliefs were that they themselves would learn best from simultaneous presentations. Students did learn best from simultaneously presented representations of mitosis and meiosis, but only when paired with self-explanation prompts to discuss the relationships between the graphics. These results provide new recommendations for helping students draw connections across visual representations, particularly mitosis and meiosis, and suggest that science educators would benefit from shifting their teaching beliefs to align with beliefs about their own learning from multiple visual representations.

INTRODUCTION

Science, even in a defined field of study such as biology, is not a set of discrete facts, but an interconnected system of complex concepts. The development of conceptually organized and integrated scientific knowledge is an overarching goal of science education, articulated at the K–12 level in the United States recently within the Next Generation Science Standards ( NGSS Lead States, 2013 ). One of the most common instructional supports that science teachers use to engage students in thinking about complex relationships is sequences of visual representations such as diagrams, pictures, or animations ( Roth et al. , 2006 ). Visualizations are central to the learning of science, the practice of science, and the communication of science, and both cognitive scientists and educators agree that they are a vitally important component of science teaching ( Ainsworth and Newton, 2014 ; Eilam and Gilbert, 2014 ; Matthewson, 1999 ). Multiple representations can help learners construct deeper understanding of scientific concepts or system structures than single representations used alone ( National Research Council, 2012 ; Ainsworth, 2014 ). These may be particularly useful when concepts are complex and interrelated, as is the case when one concept builds on another. As learners compare and contrast graphics, they are better able to construct deeper domain understanding ( Ainsworth, 2014 ).

At the same time, one representation is rarely adequate to capture the entirety of a science concept, and multiple representations are often used to describe related aspects of a system (e.g., a diagram of a heart and a diagram of a circulatory system; see Roth et al. , 2006 ). Concerns have been raised, however, about the potential of multiple representations to overly tax learners’ cognitive resources, leading them to not be able to fully process or reason on the basis of the information provided or notice and make inferences about the relationships between the representations ( Kirschner, 2002 ; Cho et al. , 2007 ). This is particularly problematic when the relationship between the representations is important, such as in the case of the model of a heart and a model of the human circulatory system with the heart at its center. Similarly, when two diagrams are intended to allow learners to compare and/or contrast aspects of systems, for example, comparing diagrams of cellular reproduction between those undergoing mitosis and meiosis, both visual representations capture important information that should be learned, but the differences between them are also illustrative and conceptually important.

Adding to the complexity of using multiple representations in classrooms are teacher and student beliefs about how people learn best from these tools. Teachers hold private beliefs about subject matter, teaching, and learning, and these influence their teaching practices ( National Academy of Sciences, 2000 ). Learners also have beliefs about how they learn best, though they are not always reliable judges of their own learning from different classroom practices. This is particularly true when individuals are required to put forth extra effort in learning ( Deslauriers et al. , 2019 ). Understanding these naïve belief systems and how they align with instructional outcomes may inform the development of more effective practices for using multiple representations in classrooms.

This article draws on the cognitive science literature to provide a novel lens for understanding the challenges inherent in learning biological science from the relationships between representations. Cognitive scientists have widely demonstrated that the human attentional system has limited resources, such that one can only meaningfully and actively process a limited set of information at once (see Baddeley and Hitch, 1974 ; Engle, 2002 ; Miyake and Friedman, 2012 ; Diamond, 2013 ). Generally, this system has been described as a system of executive functions (EFs), which are comprised of three broad components that enable a person to selectively allocate attention to information in the world and that are correlated but distinct processes ( Miyake et al. , 2000 ; Miyake and Friedman, 2012 ). Working memory (WM) is defined most broadly as the ability to hold information in mind and manipulate it (see Baddeley and Hitch, 1974 ; Engle, 2002 ; Miyake and Friedman, 2012 ). Working memory is not simply the ability to hold information in mind (e.g., a list of vocabulary words) but also to do cognitive work with that information (e.g., reorganizing new vocabulary words into a concept map). Inhibitory control (IC; see Diamond, 2013 ) is described as the integrated processes of inhibiting attention and prepotent actions based on irrelevant or misleading information (e.g., saying “night” when presented with a diagram of a sun, or expending effort to not consider the size of a textbook drawing of a cell to avoid misconceptions that cells are visible to the natural eye). Task switching refers to the processing involved in changing one’s goal-oriented task engagement and routines, for example, switching categorization criterion or switching from pointing to parts of a cell diagram to explaining how cells are part of a reproductive system (see Miyaki et al. , 2000). While separate processes, EFs are generally believed to share a limited set of resources, such that if someone is exerting all of their cognitive resources attempting to inhibit attention to something very salient but misleading, they will have less capacity to use WM to make inferences about the relationships between representations.

EFs are primarily controlled by the prefrontal cortex of the brain, an area that has been found to develop well into adolescence ( Diamond, 2013 ). Therefore, when reasoning about complex relationships between visual representations, which has a high requirement for EFs (see Waltz et al. , 2000 ; Simms et al. , 2018 ), children may need significant support for noticing key correspondences of visual representations and ignoring irrelevant or misleading features. This is particularly the case when reasoning about complex concepts such as scientific systems or solving complex problems (e.g., Gick and Holyoak, 1980 , 1983 ; Zook, 1991 ).

Though using diagrams, charts, pictures, and models is common classroom practice, careful consideration of the psychological processes of learning will aid educators in optimizing student learning from these visual representations. Theories of learning from multiple representations have generally focused on how to engage cognitive resources effectively and avoid high demand that is not intrinsic to the conceptual aspects of the intended task ( Sweller et al. , 1998 ; Mayer, 2019 ). Theories of multimedia learning build on this foundation to describe how learners make sense of text and pictures presented together. While well-designed instruction that includes text and media together may enhance learning, the processes whereby learners make sense of both textual and visual input within the cognitive architecture are complex (for a full discussion, see Mayer, 2019 ) and involve the coordination of more than one cognitive subsystem ( Schnotz and Bannert, 2003 ; Schnotz, 2019 ). Complementary representations, which are designed to highlight comparisons across diagrams, as in the case of the related processes of photosynthesis and respiration, may be particularly difficult for learners to process ( Ainsworth, 2014 ).

The Role of EFs in Making Sense of Visual Representations

Cognitive scientists broadly agree that the complexity and amount of information to be processed in visual representations can be cognitively demanding, particularly when the relationships between representations are meaningful and will lead the learner to build a broader understanding of the concept being represented ( Phillips et al. , 2010 ). The optimal way to reduce the burden on learners has been explored, yet not fully answered, and the relationship between research and practice remains complex ( Ainsworth and Newton, 2014 ).

One field of cognitive science focuses on relational and analogical reasoning, exploring how reasoners draw connections between representational systems (see Holyoak, 2012 ). Some have argued that the process of drawing structural (or conceptual) relationships between representations imposes a high burden on WM and IC of attention, particularly when the representations are not visible simultaneously (see Cho et al. , 2007 ; Krawczyk et al. , 2008 ; Begolli et al. , 2018 ).

One can compare the surface and structural elements of visual representations. Surface-level elements are those that are based in the appearance of the figures (e.g., the colors, shapes, and sizes of the objects). The structural or relational elements are the relationships between and among the visual forms, which are more typically the abstract scientific processes being explained (e.g., cell reproduction in a diagram of mitosis). As understood within the structure-mapping theory of analogical reasoning, reasoners make inferences by taking a mental model of the key structured relationships within one representation and aligning them with the key structures within a target problem, concept, or representation. They may notice the surface appearance of the visual representations, and sometimes those provide clues about how to align and recognize abstract relationships across the representations, but those surface features are typically not intended to be what was memorized. Instead, learners should map correspondences between those aligned representations to notice key abstract/conceptual similarities or differences and then draw inferences based on those alignments about the target context.

When the related representations are presented one at a time—serially—this alignment process is more effortful. When looking at the second figure, the reasoner must recall a prior visual representation and hold it in WM, while manipulating it to determine its relevance to the currently visible visual representation. The complexity involved in this mental processing is clear when considering related science concepts such as mitosis and meiosis, where a learner might align the structures of chromosomal replication, the process of cellular division, the characteristics of daughter cells, and so on, to construct an understanding of how the two types of cellular reproduction relate to each other. The learner may first notice that representations of mitosis and meiosis often both contain circles that show cells, and each cell has some wiggly lines (chromosomes) inside (i.e., the surface features of these representations), but this is not the key insight; rather, learners must go further to notice the relationships . They must see the changes from one cell to the next and the relative numbers of chromosomes in particular.

Importantly, the relational reasoning literature suggests that having visual information available should reduce the burden on reasoners’ cognitive processing by providing WM off-load, yet at the same time, this may not be enough to ensure that reasoners notice and map correspondences across representations ( Gick and Holyoak, 1983 ). Thus, reasoners may require additional support to draw their attention to link the representations actively. This may be especially important when the burden on EF for representing the information is high ( Richland and McDonough, 2010 ; Begolli and Richland, 2016 ). In a classroom context, children’s individual level of EF capacity predicted learning from a lesson in which visual representations were not visible simultaneously ( Begolli et al. , 2018 ), providing some evidence that, particularly for children with lower levels of EF available, ensuring that multiple representations are visible simultaneously and well supported might be important for improving learning.

In contrast, cognitive scientists within the field of multimodal learning, drawing on Baddeley and Hitch’s (1974) model of WM, as well as cognitive load theory (see Sweller et al. , 1998 ), have argued that simplifying representations is important to reducing cognitive overload and improving intended learning (for a comprehensive review of historical visual representation research, see Phillips et al. , 2010 ). In particular, Mayer’s cognitive theory of multimedia learning (see Mayer, 2019 ) suggests that, when a person processes a visual representation with text, he or she develops two mental representations of the material. One mental representation draws on resources within the verbal WM system based on the text, and the other draws on resources within a visual–spatial WM store, which Baddeley’s model of WM suggests draws on the same overall WM resources. So, if a visual representation has too much information of both types, a learner’s attentional resources might be overloaded, leaving little EF stores to inhibit irrelevant information (e.g., irrelevant colors and graphics to promote interest). More importantly to our current discussion, this might also impede processing of the relationships between representations. The theory does hypothesize that providing language and visual representations simultaneously can improve overall knowledge, but suggests that auditory narration is better than written text when possible. Thus, it is not clear how one would optimize learning from multiple representations when the instructional goal is to have students compare these, recognize correspondences, recognize differences, or otherwise relate them. The field of multimodal learning and cognitive load suggests that presenting two visual representations with accompanying text labels, all simultaneously, might be too high a burden for the EF system.

Thus, while science educators generally agree that guiding students toward an organized, complex, richly connected understanding of science topics through multiple visual representations is desirable ( Ainsworth and Newton, 2014 ), implementing this type of instruction requires careful consideration of the cognitive processes involved, and the implications for sequencing presentation of such representations are not yet clear. Classroom instruction that effectively supports children in making deep conceptual connections is challenging for teachers (e.g., Stein et al. , 2008 ; Smith et al. , 2009 ), so more information about these details of instruction have important applied implications.

The Current Studies

The studies reported in this article explore teachers’ and learners’ intuitive beliefs about learning from visual representations of related science concepts (Study 1) and tests those beliefs experimentally in a computer-based classroom lesson with middle school students (Study 2). Learning from multiple representations is a key pedagogical consideration in teaching science for reasoning ( NGSS Lead States, 2013 ) and raises questions about whether optimal learning emerges from two representations displayed simultaneously, where EF resources could be focused on drawing connections and generating inferences, or displayed serially, where EF resource demands on processing each representation would be reduced.

This study focused on the two processes of cellular reproduction: mitosis and meiosis. This is a critical component of most introductory biology curricula and one that every high school student must master ( NGSS Lead States, 2013 ). While mastery of these topics requires more depth than introduced here, this is an ideal pedagogical context for examining intuitions and learning from multiple representations, as these two processes are highly related but have core structural differences that are regularly made visible through diagrammatic representations. Thus, the findings here will have direct implications for teaching this core biological topic and also will provide insights for any of the many pedagogical contexts in the sciences where representations are necessary to support students in building from the understanding of one case to a second with related but different structure. This might include comparing representations of plant and animal cells or, at a more complex level, anatomical systems such as respiratory and circulatory systems.

The first study explored implicit beliefs of pre-service teachers, adult non-educators who were currently undergraduates, and middle school children, regarding whether people learn better from multiple visual representations that are presented simultaneously or serially and the reasons they gave for holding these beliefs. The three groups were asked how they themselves learned best, as well as how they believed a younger set of learners would learn best to distinguish between their understanding of their own cognition and their beliefs based on their own naïve theories of pedagogy. The primary research questions were whether these two sets of learning beliefs would diverge, assessing whether there were differences between peoples’ beliefs about their own learning and their beliefs about pedagogy—how others would learn best.

The second study tested how these implicit beliefs related to the mental representations children gained from instruction that involved comparisons between multiple representations with lower or higher levels of support for drawing connections between them, including serial versus simultaneous presentation, as well as more explicit prompts to actively align and compare the representations. To answer these questions, we developed an experiment wherein children were randomly assigned to learn from a computerized instructional module in which only the method of presenting diagrams varied across conditions.

The preponderance of research regarding how people process multiple representations has been conducted with adults rather than children ( Cook, 2006 ). This is an important oversight, given that visual representations are the most commonly used instructional supports in American K–12 science classes ( Roth et al. , 2006 ). This study aimed to test predictions made by the cognitive science literature regarding best practices for supporting students in learning from multiple representations. We tested both the mode of ordering the representations (sequentially vs. simultaneously), as well as the level of pedagogical support provided. Thus, Study 1 and 2 together allowed us to gain insight into adults’ and youths’ teaching beliefs about ordering multiple representations and the alignment between these beliefs and students’ actual learning.

Study 1: SURVEY OF IMPLICIT BELIEFS ABOUT LEARNING FROM MULTIPLE REPRESENTATIONS

While both cognitive scientists and science teachers agree that visual representations are important tools in teaching science, the alignment of research to teaching practice is not always direct, and in the demands of real classroom practice, teachers rely heavily on personal judgment in deciding what visual representations to use and how they will be presented ( Ainsworth and Newton, 2014 ). Yet little is known about what informs these judgments, particularly when it comes to multiple representations of related science concepts.

One concern that teachers have is about student competency in interpreting visual representations ( Eilam and Gilbert, 2014 ). But students are not blank slates when they approach a diagram or picture. They have their own metacognitive beliefs about how they learn best. Students, however, are not always the best judges of what helps them learn, particularly when it comes to passive versus effortful learning ( Deslauriers et al. , 2019 ).

The metacognitive beliefs people hold can influence what and how they can learn ( Greeno et al. , 1996 ; Pamuk et al. , 2016 ). Teacher beliefs about learning can influence their classroom practices of teaching, and in turn, indirectly affect student achievement ( Muijs and Reynolds, 2015 ). Further, beginning science teachers and experienced science teachers can have different views of their role in helping students learn. Beginning teachers may hold more teacher-centered traditional views of delivering information ( Luft and Roehrig, 2007 ). Part of building learning theory in educational contexts involves understanding the teaching and learning beliefs that teachers bring to the classroom. Here we collect data to understand beliefs about teaching and learning expressed by pre-service teachers, adult non-educators, and children. We specifically focus on the pedagogical context of how to best sequence visual presentations of multiple representations—simultaneous versus serial presentation.

Participants.

The survey sample included 89 pre-service teachers, 211 adult non-educators, and 385 middle school children. The 89 pre-service teachers were enrolled in a combined credential/master of arts in teaching program at a large suburban university. They were enrolled in a basic cognition class but had not yet received any explicit instruction about either EFs or the use of visual representations. The 211 adult non-educators were undergraduate students at the same university and represented a wide variety of different majors. All adult participants consented to participation in the study. The 385 middle school students were seventh-grade students of three science teachers at two different suburban schools from the same upper-middle-class district. The middle school participants were recruited through their science teachers. The day before the study, students were given a letter to take home that described the study. The letter informed parents/guardians that students were not required to participate, and that no student-identifiable data would be collected. On the day of data collection, students were read a description of the study and indicated assent through the raising of hands. Students were informed that they could remove themselves from the study at any time and receive the same instruction through text-based instruction provided by their teacher. One student opted out of the study and was not included in further analyses.

Materials and Procedure.

The surveys administered to each population were slightly different in framing due to the different educational background of the children, adults, and teacher candidates, but the key questions analyzed in this study were the same. Also, we intentionally asked each population about their beliefs of how a younger population would learn best, so this differed across participant groups.

Adult participants were given a pencil-and-paper survey as part of a larger, unrelated study. Participants were told that their responses would be used to inform development of new science teaching materials. They were asked about their own science backgrounds and whether they felt they could describe the related processes of mitosis and meiosis to a friend. They were then given a forced-choice item that asked whether they thought they would learn better from a combined (simultaneous presentation) diagram of mitosis and meiosis, or if they thought it would be better to learn from separate diagrams (presented serially). A free-response question asked them to justify their choices. A follow-up question asked them to predict how a middle school student would learn better, and again, participants were asked why they made that choice. The entire survey took approximately 5 minutes to complete, and the key questions are available in the Supplemental Material.

Middle school students answered the same basic questions as the adults, but as they had not yet been exposed to instruction about mitosis and meiosis, the forced-choice item asked whether they would prefer to learn about the related topics of animal and plant cells through representations of each cell presented simultaneously or serially, and why. This was followed up by items asking how they thought a fourth-grade student would learn best and why.

Measures and Data Coding.

Forced-choice responses asking how people would best learn from related diagrams were simply coded “simultaneous” or “serial.” For free-response items asking participants why they made the choice they did, categorical codes were developed to quantify data for comparison. These codes were developed through an iterative process informed by EF literature on learning and its relationship to comparing and contrasting representations (e.g., Krawczyk et al. , 2008 ; Holyoak, 2012 ; Begolli et al. , 2018 ) and refined by the responses themselves. Interrater reliability across codes was set at kappa ≥0.80. Two raters coded a training data set of student responses to allow for discussion and resolution of any discrepancies in codes assigned. The two raters then independently coded 20% of each data set to attain reliability. A single rater (J.H.) coded the remaining data independently. The coding manual is available in the Supplemental Material.

Four codes scored responses to items that asked why people would learn best from either simultaneous or serial presentation of visual representations. These codes were 1) ability to compare and contrast; 2) promotes deeper understanding; 3) described as easier or not as difficult; and 4) cites reducing confusion as a goal. The codes were not mutually exclusive, and a response could receive more than one code. The differences between each of these codes rested on the participants’ explicit use of words that highlighted each of these ideas (e.g., to compare or to reduce confusion) or a clear framing that allowed a coder to differentiate their intention. “Easier versus difficult” provided only a graded description of difficulty and was coded separately from any statements regarding confusion as a mechanism that would form the source of any difficulty. These codes (reported with their associated kappa statistics) are detailed in Table 1 .

Beliefs results are provided for the three sets of participants separately, with overall means and results shown in Figure 1 . Each is discussed in turn, analyzing frequency of endorsing simultaneous versus serial representations both for their own learning and for teaching others who were younger than themselves.

FIGURE 1. Adult, pre-service teacher, and middle school student beliefs about optimal presentation of multiple representations for one’s own learning versus instruction of others.

Pre-service Teachers.

Pre-service teachers ( n = 89) did not overwhelmingly endorse one way of presenting conceptually related visual representations for their own learning, with beliefs split between simultaneous and serial presentation orders as optimal. This difference was not significant; χ 2 (1, N = 89) = 0.91, p = 0.34. Interestingly, their reasons for selecting each of these two different orders were different. Among teachers who endorsed serial presentation of related diagrams for their own learning, avoiding confusion was the most often cited reason ( n = 17, 34.7%). Those who said they preferred related diagrams presented simultaneously cited the ability to compare and contrast as the reason this was desirable ( n = 34, 85.0%).

When asked how middle school students would learn best, however, pre-service teachers significantly often changed their beliefs, and indicated that the learning needs of middle school students differed from their own; χ 2 (1, N = 89) = 22.48, p < 0.01. As shown in Figure 1 , most indicated that serial presentation would be optimal for middle school students. This suggests that the pre-service teachers held a tacit belief that there is a developmental difference in the learning needs of middle school students versus adults when analyzing multiple representations.

The reasons preservice teachers gave for these decisions are similar to those described for how they would themselves learn. Of the 59 respondents who said middle school students would learn best from serially presented diagrams, the most-cited reasons for endorsing this style were avoiding confusion ( n = 22, 37.2%), ease of interpretation ( n = 15, 25.4%), and promoting deeper understanding ( n = 11, 18.6%). Of the 30 pre-service educators who said middle school students would learn best from simultaneously presented diagrams, 80.0% ( n = 24) cited the ability to compare and contrast as the reason why this method was preferable.

Adult Non-educators.

In contrast to pre-service teachers, the adult non-educators, who were currently undergraduate students, indicated a clear preference for simultaneous presentation for their own learning; χ 2 (1, N = 211) = 50.28, p < 0.01. Again, the reasons for endorsing simultaneous ordering were the same. Among the 54 participants who endorsed serial presentation for their own learning, the most common reasons cited were ease of interpretation ( n = 23, 42.6%), and avoiding confusion ( n = 22, 40.7%). Of the 157 people who preferred simultaneous presentation, 89.2% cited the ability to compare and contrast as the reason ( n = 140).

When asked whether serial or simultaneous visual representations were preferable for middle school students, like pre-service teachers, a significant number of adult non-educators felt middle school students would benefit from a different manner of presentation than themselves as adults; χ 2 (1, N = 211) = 21.96, p < 0.01. This shifted to more recommendations for serial presentation for children than for themselves, though there were not significant differences between these two; χ 2 (1, N = 211) = 3.99, p = 0.05.

The main reason one style of presentation was preferred over the other was similar for the adult non-educators as for the pre-service teachers. For those endorsing serial presentation, avoiding confusion was cited by 48 of the 91 respondents (52.7%). Among 120 people who felt simultaneous presentation would be better, 89 (74.2%) cited the ability to compare and contrast as important.

Middle School Students.

Like their adult counterparts, middle school students had strong opinions about the way related visual representations should be presented. Students of color in the proportion of participation as a result of the transition online. As shown in Figure 1 , simultaneous presentation was preferred for their own learning. This difference was significant; χ 2 (1, N = 385) = 124.57, p < 0.01. Again of interest is that these youth cited the same reasons for preferring simultaneous versus serial presentation order as the adults did. Those who preferred serial presentation endorsed its role in avoiding confusion (46.9%, n = 39), and the belief that it would lead to greater understanding (24.1%, n = 20). The significant reason for endorsing simultaneous presentation was the ability to compare and contrast (86.4%, n = 261).

As in adults, the children’s beliefs about how they would learn best differed significantly from how they thought younger students would learn; χ 2 (1, N = 384) = 19.02, p < 0.01. When asked what presentation would be better for fourth-grade students, the middle school students were split, with 197 (51.3%) endorsing simultaneous presentation. Respondents who selected serial presentation were more likely than those selecting simultaneous presentation to say it would help younger children avoid confusion or distraction (12.8%, n = 24) and lead to greater depth of understanding (7.0%, n = 13). Those who selected simultaneous presentation were most likely to suggest that the ability to compare and contrast would be enhanced (23.3%, n = 46).

Importantly, the beliefs of middle school students about how they would learn best differed significantly from the beliefs pre-service teachers held about the students’ learning needs. Middle school students strongly preferred simultaneous presentation for their own learning, but the pre-service teachers felt that the students would learn better from serial presentation; χ 2 (1, N = 474) = 68.94, p < 0.01.

Previous research has indicated that beliefs about learning are important for their influence on teacher instructional practice ( Friedrichsen et al. , 2011 ). Collected survey data from the 211 adult non-educator and 385 middle school student samples were fairly consistent, in that both groups preferred to learn from simultaneous presentation of visual representations when learning about conceptually connected science concepts. The 89 pre-service teachers differed in that they did not significantly choose one manner of presentation over the other for their own learning. All of the adults were more likely to prefer serial presentation for middle school students, though the students themselves strongly preferred simultaneous presentation. This difference was significant when comparing pre-service teacher preference for middle school student learning and the middle school student preferences.

For participants who were drawn to serially presented visual representations, concern about the amount of information to be processed was commonly expressed. A typical response reads, “With just one [simultaneously presented diagram] it might get jumbled together and confusing.” Those who preferred simultaneous presentation were more likely to cite the ability to compare and contrast as being desirable. This suggests that, across all three groups, participants had a sense that the EF resources required to process simultaneously presented diagrams would be much higher, at least initially, than the cognitive demand of processing serially presented diagrams.

Pre-service teachers felt strongly that middle school students needed serial presentation of diagrams of conceptually related content in order to learn best, while the responses of non-educators were evenly split between endorsing serial and simultaneous presentation. As one teacher pointed out, “Two diagrams would keep each process separate. This would help students get a clear idea of both processes before they are shown together.” Non-educator adults and middle school children were mixed on what method of presentation children younger than themselves would need, and the difference did not rise to the level of significance. This indicates that the pre-service teachers held stronger beliefs that developmental processes underlie the ability to process complex science diagrams.

One interesting area where pre-service teachers and middle school students disagreed was on how middle school students would learn best from multiple visual representations. While the pre-educators felt students would need serial presentation, 78.4% of students preferred simultaneous presentation and the ability to compare and contrast across related representations shown together. This mismatch between the beliefs that pre-service teachers held about student learning, and the students’ own metacognitive beliefs may signal misunderstandings about learner capabilities. Pre-service teachers appear to take a cautious view of the limits of the EFs of students as they grapple with complex diagrams, while students may overestimate their abilities to make meaningful connections between related science representations.

A second study was designed to examine how different presentation styles affected student learning. The results of that study are summarized in the next section.

Study 2: EXPERIMENT VARYING PRESENTATION OF VISUAL REPRESENTATIONS IN A MIDDLE SCHOOL LESSON ON MITOSIS AND MEIOSIS

While the survey results of Study 1 suggest that both adults and children have deeply held beliefs about the ways students learn from conceptually connected visual representations, the literature is not clear on how these beliefs align with actual learning outcomes. The second study provides data on student learning from two representations aligned in different ways. This study compared not only serial versus simple simultaneous diagram presentation, but also added two simultaneous presentation conditions suggested by cognitive scientists interested in EFs: simultaneous presentation with support for noticing and simultaneous presentation with structure mapping support.

The students in Study 1 also participated in Study 2 and were recruited through three seventh grade science teachers at two suburban schools. Both schools were from the same upper-middle-class district. Two of the teachers were from school A, and across their eight classes, they taught 224 of the study participants. The teacher at school B had five classes and a total of 161 study participants. Due to course work planning constraints of the teachers, researchers had only 1 day to collect data. Though no individual demographic data were collected, the students in the study group were described by participating teachers as representative of the school population, as summarized in Table 2 .

The day before the study, students in all classes were given a letter to take home that described the study. The letter informed parents/guardians that students were not required to participate, and that no student-identifiable data would be collected. On the day of data collection, students were read a description of the study and indicated assent through the raising of hands. Students were informed that they could remove themselves from the study at any time and receive the same instruction through text-based instruction provided by their teacher. One student opted out of the study and was not included in further analyses.

This study was completed under the IRB approval of the University of California, Irvine, HS no. 2012-9111.

Materials and Procedure

Instructional lesson..

A computer-based instructional module was designed using the Web-based Inquiry Science Environment. The students first responded to a survey (described in Study 1) that asked them how they thought they would learn best from related diagrams. This was followed by a lesson that introduced the related concepts of cell replication and reproduction through mitosis and meiosis. The module forced students to complete learning tasks on each screen before moving forward. After advancing the module, they were not able to move backward. This ensured that students completed all steps of instruction in order.

Regardless of the method of presenting diagrams, the text of the lesson itself remained constant and was based on the printed life sciences textbook used by seventh-grade classrooms throughout the school district ( Padilla, 2007 ). Five screens were included in mitosis instruction, one each for interphase, metaphase, anaphase/telophase, and cytokinesis. This aligned with the textbook presentation of the same material. Each screen included a diagram alongside the text. A sample of the instructional diagram for mitosis is provided in Figure 2 .

FIGURE 2. Mitosis diagram from instructional model.

After completing the mitosis instruction, students were given a constructed-response item that asked them to recall information from the lesson. This page did not include any diagrams, only a box that simply asked, “How would you describe the process of mitosis to a friend? Describe as many steps as you can.” At the completion of this screen, students saw a graphic that praised them for their hard work.

The second segment of instruction introduced the concept of meiosis through a series of seven different screens: Introduction; Interphase; Prophase I; Metaphase I; Anaphase I and Telophase I; Cytokinesis I; and Meiosis II. These segments were designed to closely align with the mitosis screens in the module. The instructional text was adapted from the ninth-grade science textbook from the same publisher as the seventh-grade textbook ( Miller and Levine, 2011 ). Some of the text was simplified to eliminate vocabulary to which the students had not yet been exposed and to match the instruction in the mitosis portion of the module. The meiosis diagram that appeared in the instructional module is shown in Figure 3 .

FIGURE 3. Meiosis diagram from instructional model.

Once students completed the meiosis instructional module, they again received a recall item on a screen containing only text. Similar to the prior recall item, students were asked how they would describe meiosis to a friend, describing as many steps as they could. Upon submission, students were provided a screen praising their hard work and their completion of this section.

The experiment sought to test whether students’ beliefs about learning from multiple representations were aligned with their patterns of learning from multiple representations. Specifically, the learning context was knowledge gain from conceptually related science diagrams rather than different diagrams of the same concept. The experimental manipulations therefore involved providing diagrams organized in four different ways within the lesson: 1) serial presentation of separate mitosis and meiosis diagrams; 2) simultaneous presentation of the diagrams side by side; 3) simultaneously presented diagrams that signaled the learner to key similarities and differences; and 4) simultaneously presented diagrams with support for structure mapping. Computer-generated random assignment to experimental condition was achieved within each classroom using dummy codes for each student, such that the researchers did not know which student was assigned which code or experimental condition. The classrooms were all existing, mixed-ability classes. Random assignment at the student level allowed us to minimize any effects of classroom teacher or classroom-level characteristics and maximize ecological validity, as the instruction took place in a whole-classroom setting with peers and everyday social context. Written materials are provided in the Supplemental Material.

Serial Presentation .

In the serial presentation condition, a mitosis diagram (see example in Figure 2 ) was provided to learners during all instruction related to learning about mitosis. A diagram of meiosis (see example in Figure 3 ) was provided to learners during all instruction related to meiosis. Diagrams were never shown on screen at the same time. No additional supports were provided. Serial presentation was included in all classrooms studied ( n = 128).

Simultaneous Presentation .

In the simultaneous presentation condition, a combined diagram showing mitosis and meiosis side-by-side ( Figures 2 and 3 with initial cells aligned side-by-side) was shown during all instruction. Therefore, when students were reading text about mitosis, they could also see the diagram for meiosis, and vice versa. There were no additional supports for noticing or interpreting the diagrams. Simultaneous diagrams were presented in all classrooms studied ( n = 124).

Simultaneous with Signaling .

In the simultaneous with signaling condition, students received the same combined diagram as in the simultaneous condition. The only difference was the addition of signaling prompts highlighted in red within the diagrams. These signals were designed to alert students to key features of the diagrams. For instance, when a diagram with instruction on cytokinesis was shown, red text asked, “Do the daughter cells look like the parent cells?” (see Supplemental Figure S1). This signaled learners to attend to an important phase in cell division that leads to miotic daughter cells that are identical to parents, while in meiotic cells, the daughter cells are each unique. Simultaneous diagrams with signaling were only offered at school A, with two teachers ( n = 80).

Simultaneous with Structure Mapping Support.

A fourth condition, simultaneous presentation with structure mapping support, draws on prior research that suggests that learners are better able to reason about representations with support (e.g., Gick and Holyoak, 1980 , 1983 ) and better able to generalize their learning when actively participating in mapping the comparative relationships ( Richland and McDonough, 2010 ). In this condition, learners received the screen that presented mitosis and meiosis simultaneously. However, before students left each instructional page, a mouse click would call up a question with a response box. For instance, the meiosis cytokinesis page read, “Take a close look at the picture, comparing the end of mitosis with the end of meiosis. In your own words, describe what is created by meiosis.”

Active generation, or testing, is known to facilitate memory and retention (e.g., see Roediger and Karpicke, 2006 ), which suggests that by having students specifically generate alignments and comparisons, one can facilitate this learning. Similar to the signaling condition, students were alerted through highlighted text to key similarities or differences between the diagrams. But in addition to having their attention guided to the important element (signaling), students were asked to actively reason about what they were noticing, identifying the relationship between the diagrams themselves. Simultaneous diagrams with structure mapping support were only offered at school B ( n = 53).

Outcome Measures and Data Coding

Outcome measures were derived from the free-response data written by students in response to prompts requesting students to describe mitosis and meiosis after instruction. This was designed to allow for a more nuanced understanding of the mental models of these systems that were developed by students, rather than simple accuracy rates in response to smaller, more explicit questions. Participant responses were downloaded directly from the teaching module into spreadsheet format for coding. Categorical codes were developed to quantify qualitative data coded by highly trained coders. At least two coders (including JH) independently scored 20% of the data, yielding above 80% agreement (high to acceptable rates of agreement) using Cohen’s kappa to control for chance reliability.

Descriptor codes for describing mitosis and meiosis were based on instructional text and iteratively refined through comparison to student responses at the development phase. Codes were derived from key principles within the biology of mitosis and meiosis, as well as characteristics of cognitive work that were predicted by the literature to indicate deep thinking, such as drawing connections and making inferences.

At the conclusion of mitosis instruction, all participants were asked to respond to the following prompt: “How would you describe mitosis to a friend? Fully describe as many steps as you can.” Eight separate features when describing mitosis were identified. These features, along with their interrater reliability (kappa) score, are listed in Table 3 .

Additional codes were added for “9: misconception” (e.g., “the male sperm and female egg meet”; K = 0.95), and “10: identification of surface features” (K = 0.81) such as size (e.g., “The process keeps … dividing into smaller parts”), color (e.g., “attached to a yellow string”), or nonspecific use of diagram labels (e.g., “It goes through interphase, prophase, metaphase, anaphase, telophase, and cytokinesis”) or references stages (e.g., “I learned that mitosis is a process that has lots of steps to the cycle”) as the whole response.

At the end of meiosis instruction, all participants were presented with a constructed-response item that asked: “How would you describe meiosis to a friend? Fully describe as many steps as you can.” Student responses mentioned 10 different structural features of meiosis, shown in Table 4 , along with their interrater reliability (kappa) statistic.

As with mitosis, coders scored when students mentioned surface features (K = 1.0), like size or color, or simply listed names of phases rather than describing them. Some students mistakenly described the cell replication process of mitosis when responding to the “describe meiosis” prompt, and these responses were coded separately as well (K = 0.90).

Principal Component Analyses.

Principal component analysis for categorical data of the characteristics of mitosis and for meiosis was used to identify underlying patterns of responses. These analyses were completed using the Statistical Package for Social Sciences (SPSS) software. Principal component analysis was appropriate, as all data were categorical. Direct oblimin rotation was applied. An oblique rotation was preferred, as the individual components all refer to parts of the same process of cell division, and therefore correlation among variables was expected. Each component met the Kaiser criterion (Kaiser, 1960) for selection with an eigenvalue greater than 1.0. Component loadings greater than 0.40 were retained.

Students’ free responses to the “describe mitosis” and “describe meiosis” prompts provide data not only about student understanding of each process, but also on their inference errors across conditions.

Mitosis Free-Response Analyses.

All cell features and cell processes noted by students were included in the principal component analysis. The descriptors clustered into three factors: rich description , which explained 29.05% of variance in the data; simple description , which explained 14.62% of the variance; and surface-level description , which explained 12.64% of total variance. Taken together, these factors explained 56.32% of variance in participant responses. The individual component loadings are described in Table 5 .

a Component loadings < 0.40 are suppressed.

b Variable principal normalization.

Rich responses, factor 1, meant that participants discussed several of the key features of mitosis and highlighted the role of spindle fibers and chromosomes. These can be contrasted with simple responses, factor 2, which were responses that focused primarily on the cell growing and dividing, with little additional meaningful detail. Further, identification of identical cell creation as a feature of mitosis was negatively correlated within a simple response (see Table 5 ). Surface-level descriptions, factor 3, showed reliance on colors, shapes, sizes, or the use of labels without describing the process of replication or the creation of identical cells. These are important, because they reflect responses that are purely descriptive of the appearance of the diagrams and fail to engage in the abstract structure that is key to cell reproduction. These are responses that suggest the learner has not engaged in the higher-order, relational thinking that was intended in the instruction. Examples of each type of response are shown in Table 6 .

Regression scores for each component were obtained using SPSS and were then compared across conditions using a one-way between-subjects analysis of variance (ANOVA). There was no significant differences in the distributions of either rich description, F (3, 353) = 0.34, p = 0.80, or simple description, F (3, 353) = 1.33, p = 0.27, across condition, but there was an overall significant effect for surface-level description by condition, F (3, 353) = 5.26, p = <0.01.

To further understand group differences for relying on surface features in descriptions of mitosis, we conducted a series of t tests for independent means of the regression scores. Participants in the signaling condition (M = 0.24) were more likely to rely on surface features than those in the serial, M = −0.08, t (193) = −2.86, p = <0.01, or support for structure mapping, M = 0.00, t (119) = −0.24, p = <0.01, conditions. Further, participants who received support for structure mapping outperformed those who saw the diagrams combined with no support, M = 0.03, t (160) = 2.4, p = 0.02. These results are summarized in Table 7 .

*Denotes significant difference, p < 0.05.

Meiosis Free-Response Analyses.

The same analysis was repeated for free-response data to a meiosis prompt. At the end of meiosis instruction, all participants were presented with a constructed-response item that asked: “How would you describe meiosis to a friend? Fully describe as many steps as you can.” A total of 320 participants responded to this prompt, and this was coded as described in the study 2 Methods section.

Student responses were coded for mention of 10 different structural features of meiosis and for whether they made errors confusing meiosis for mitosis. These features are described more fully in Table 4 .

These data were analyzed using principal component analysis to see whether and how each coded descriptor would contribute to overall patterns of responses. A total of three were identified: rich description, simplistic description, and confused with mitosis.

The three factors together explained 53.02% of variance in responses. Rich description explained 27.87%; simplistic description explained 13.25%; and confusing mitosis with meiosis explained 11.90% of variance in participant responses. The individual factor loadings are described in Table 8 .

As with the “describe mitosis” prompt, we provide examples of the three patterns here. A rich description in answer to the “describe meiosis” prompt was associated with noting several different structural features of the replication process. A simplistic description might correctly identify meiosis as a process of cell division, but little else was fully described. While first and second cell division might be included, little more detail was described. The third type of response identified by components analysis was one that confused mitosis with meiosis. Although the student may correctly identify cell growth, the defining factors of meiosis are not described. Examples of each type of response are shown in Table 9 .

Regression scores for each of the three factors were calculated and compared across experimental condition using ANOVA. There were no significant differences in the distributions of either rich description, F (3, 316) = 1.35, p = 0.26, or simplistic description, F (3, 316) = 0.07, p = 0.98, responses across condition. There was, however, a significant condition effect for confusing meiosis with mitosis, F (3, 316) = 3.43, p = 0.02.

The error component of confusing mitosis with meiosis was isolated for further analysis across the data using t tests for independent means. As shown in Figure 4 , those in the simultaneous condition with structure mapping support (M = −0.32) were significantly less likely to be in the group that confused mitosis and meiosis than those in the serial, (M = 0.00, t (133) = 2.5, p = 0.01), simultaneous, (M = −0.01), t (140) = 2.24, p = 0.03), or signaling, (M = 0.15), t (107) = 3.77, p < 0.01), conditions.

FIGURE 4. Mean component factor scores showing rates of errors confusing mitosis and meiosis across conditions.

Like their adult non-educator counterparts, middle school children felt they would learn better from simultaneously presented visual representations of related science information. And, like the adults, they described the ability to compare and contrast across diagrams as desirable.

The experimental learning data provided insight into the validity of these beliefs and a more nuanced implication for instruction. While simultaneously presented representations did enhance student ability to make sense of science information concerning related concepts, they were only optimized when they included explicit supports for actively engaging learners in making the key connections across the representations. Simply having the two related diagrams presented together was not enough to engage and sustain the higher cognitive processes of the EF system, nor was directly drawing the learner’s attention to key features of the diagrams. Middle school children did need support for the perceived benefit of comparison and contrast to be achieved

Though this experiment revealed that students in all conditions learned from instruction supported by visual representations, there were two important ways in which students in the supported structure mapping group outperformed the others. First, in describing mitosis, these students were far less likely to rely on surface features than those who received diagrams presented simultaneously either with no support or with only signaling support. They were not distracted by the number of steps, colors, shapes, or labels of the diagrams. Instead, active mapping of the correspondences appeared to draw attention to the processes rather than the drawings or the textual labels themselves.

Second, when presented with conceptually related science content, they were less likely to confuse the two processes, even though the diagrams were visible simultaneously during all instruction. This finding was robust, with the support for structure mapping group outperforming serial, simultaneous, and signaling conditions. It suggests that the students who received structure mapping support had a clearer picture of the key conceptual similarities and differences between the two cellular processes at the conclusion of instruction.

The greatest differences were seen when comparing the simultaneous presentation with signaling presentation and the structure mapping group. This is surprising, in that prior research has suggested that signaling can aid learners in identifying important aspects of complex diagrams. In the case of diagrams of related processes, however, adding only the written signals may have added too much to the overall cognitive load for learners whose EFs, without additional support, were not sufficient to handle both the simultaneous diagrams and the signals intended to direct their attention to key correspondences (for a description of other research on outcomes related to signaling, see Mayer and Fiorella, 2019 ). This is contrasted with the support for structure mapping condition, which also included additional visual input, but actively engaged learners in describing what they saw instead of simply directing them to consider a specific aspect of the diagrams. Structure mapping prompts did appear to support learner ability to make sense of the same aspect of the diagram to which cueing only drew their attention. This is particularly notable, as the presence of both diagrams simultaneously did not overwhelm the learners in the supported condition, and it also appeared to enhance IC for information irrelevant to the task at hand.

OVERALL DISCUSSION

These studies elicited privately held beliefs from pre-service teachers, adult undergraduates, and middle school children about learning from visual representations of related science concepts and compared these beliefs to learning outcomes.

As predicted by prior research on EF development, and as shown in Figure 4 , children did need support for mapping key elements across diagrams of the related science concepts of mitosis and meiosis in order to avoid surface-level understanding and errors confusing the represented ideas. But learners who received that support were able to develop more complex understandings of the key relationships between the two processes. This brings to light a possible misalignment between the beliefs of pre-service teachers, who as seen in Figure 1 , endorsed serial presentation as easier for students to understand, and the metacognitive beliefs of children, who preferred the challenge of comparing and contrasting. While the pre-service teachers may be underestimating the EF of children’s minds, children may overestimate what they can do without support.

Consistent with theory based in the relational reasoning literature (see Richland and McDonough, 2010 ; Begolli et al. , 2018 ), this study found that children were able to process and create a deeper understanding of complex, related science topics when they had support for making connections across representations. This active involvement in making connections led to a lessened reliance on surface features of diagrams. More importantly, supporting students in structure mapping across related representations led to a deeper understanding of the key similarities and differences of the science concepts described and fewer misconceptions confusing mitosis with meiosis. The conditions that included active generation during learning may also have received a boost through the generation process supporting memory itself (e.g., see Roediger and Karpicke, 2006 ).

These studies show that focusing on addressing only the limitations of WM by limiting the presentation of simultaneous visual representations may lead to missed opportunities to help learners develop more complex mental models. Providing students with simultaneous representations of related science concepts can lead to learning that relies on structural correspondences rather than featural similarities and differences, but only if adequate support for EFs is provided. That these results held true in a real instructional setting with child learners is exciting, as they suggest that, when presented with support for mapping key relations, simultaneously presented visual representations of related science concepts can help students in science classrooms develop a greater understanding of complex interconnections in science.

Limitations

The main limitation of this study was the brevity of the overall delay to test. The constrained design allowed us high control in order to examine the effects of varying the instructional order and support for presenting materials. At the same time, it will be important to follow this work with an examination of how these effects persist over time.

Additionally, researchers were only able to test three conditions in each school. Though assignment was randomized across classrooms within those schools, comparison directly across schools was not possible for every condition. This may have underestimated school-level effects. Future studies would be further enhanced by the collection of student demographic and pretest and posttest data that could not be collected in the current study.

Implications for Practice

Together, these results provide new insights into how to optimize student learning from visual representations, and they also provide science educators with an important lens through which to consider their beliefs and practices of using visualizations. Integrating the theories of relational thinking and EFs helps to clarify why teachers must go beyond simply providing multiple visual models, diagrams, or other types of representations in sequence. We can infer that students may learn the details being shown in the representations when presented serially, and perhaps retention could even be facilitated in that way by reducing the amount of information to attend to, thereby reducing the overall EF load. But to promote broader understanding of how concepts fit together or to recognize commonalities and differences, this presentation style may not be optimal. Students may struggle to align and connect the ideas from two representations and ideas presented serially, which will limit the inferences they can make and may lead to misconceptions or misunderstandings. Thus, this report demonstrates the utility of supporting students in deepening their understanding of biology concepts by simultaneously showing two representations that are intended to be compared.

To best support teachers in incorporating this into their practice, we must also take note of the beliefs data we found. As shown in Figure 1 , these data in particular suggest that teachers would benefit from being shown the distinctions between their beliefs about their own learning and their beliefs about teaching their students. We found that pre-service educators tended to believe that they learn differently than their students, which is an extremely important point for teacher education and science education researchers to consider. People’s intuitions about their own learning did mirror the results we found in favor of better learning through supported simultaneous presentation. But pre-service teachers’ intuition was to teach child learners through serial presentation of diagrams. We know that educators’ beliefs are powerfully related to practice, which means that interventions and educational reforms that do not align with beliefs can be very difficult to change (see Munby, 1984 ; Wallace 2014 ).

Rather than aiming to convince teachers that their beliefs about teaching are incorrect, it likely will be more productive to highlight how their beliefs about their own learning are more in line with student learning in this case. That being said, their teaching beliefs seem to highlight that more support can be needed for younger learners to notice and draw connections across visual representations, which is also demonstrated in our data. So the overall implication is that learning can be optimized by presenting related visual representations simultaneously, but with additional support to help learners identify the relevant correspondences and differences without overloading their cognitive systems. Adding prompts for students to discuss and connect what they notice between these visual representations was a particularly powerful strategy. This has implications for both classroom teaching and visual texts, such as textbook design.

ACKNOWLEDGMENTS

This work was conducted with support from the National Science Foundation (grant no. 32027447) and the Institute of Education Sciences, through R305A190467 to the University of Chicago and the University of California, Irvine. The opinions expressed are those of the authors and do not necessarily represent views of the institute, the National Science Foundation, or the U.S. Department of Education.

• Ainsworth, S., & Newton, L. ( 2014 ). Teaching and researching visual representations: Shared vision or divided worlds? In Eilam, B.Gilbert, J. (Eds.), Science teachers’ use of visual representations . ( Models and modeling in science education , Vol. 8 , pp 29–49). Cham, Switzerland: Springer International Publishing. Google Scholar
• Baddeley, A. D., & Hitch, G. J. ( 1974 ). Working memory . In Bower, G. A. (Ed.), The psychology of learning and motivation (pp. 47–89). New York: Academic Press. Google Scholar
• Cho, S., Holyoak, K. J., & Cannon, T. D. ( 2007 ). Analogical reasoning in working memory: Resources shared among relational integration, interference resolution, and maintenance . Memory & Cognition , 25 (6), 1445–1455. Google Scholar
• Cook, M. P. ( 2006 ). Visual representations in science education: The influence of prior knowledge and cognitive load theory on instructional design principles . Science Education , 90 , 1073–1091. Google Scholar
• Deslauriers, L., McCarty, L. S., Miller, K., Callaghan, K., & Kestin, G. ( 2019 ). Measuring actual learning versus feeling of learning in response to being actively engaged in the classroom . Proceedings of the National Academy of Sciences USA , 116 (39), 19251–19257. Medline ,  Google Scholar
• Diamond, A. ( 2002 ). Normal development of prefrontal cortex from birth to young adulthood: Cognitive functions, anatomy, and biochemistry . In Stuss, D. T.Knight, R. T. (Eds.), Principles of frontal lobe function (pp. 466–503). Oxford, England: Oxford University Press. Google Scholar
• Diamond, A. ( 2013 ). Executive functions . Annual Review of Psychology , 64 , 135–168. Medline ,  Google Scholar
• Eilam, B., & Gilbert, J. K. ( 2014 ). The significance of visual representations in the teaching of science . In Eilam, B.Gilbert, J. (Eds.), Science teachers’ use of visual representations (Models and modeling in science education, Vol. 8 , pp 3–28). Cham, Switzerland: Springer International Publishing. Google Scholar
• Friedrichsen, P., Driel, J. H. V., & Abell, S. K. ( 2011 ). Taking a closer look at science teaching orientations . Science Education , 95 (2), 358–376. Google Scholar
• Gick, M. L., & Holyoak, K. J. ( 1980 ). Analogical problem solving . Cognitive Psychology , 12 (3), 306–355. Google Scholar
• Gick, M. L., & Holyoak, K. J. ( 1983 ). Schema induction and analogical transfer . Cognitive Psychology , 15 (1), 1–38. Google Scholar
• Greeno, J. G., Collins, A. M., & Resnick, L. ( 1996 ). Cognition and learning . In Berliner, D.Calfee, R. (Eds.), Handbook of educational psychology . New York, NY: Simon & Schuster Macmillan Publishing. Google Scholar
• Holyoak, J. ( 2012 ). Analogy and Relational Reasoning . In Holyoak, K. J.Morrison, R. G. (Eds.), The handbook of thinking and reasoning (pp. 234–259). Oxford, England: Oxford University Press. Google Scholar
• Kaiser, H. F. ( 19600 ). The application of electronic computers to factor analysis . Educational and Psychological Measurement , 20 , 141–151. Google Scholar
• Kirschner, P. A. ( 2002 ). Cognitive load theory: Implications of cognitive load theory on the design of learning . Learning and Instruction , 12 , 1–10. Google Scholar
• Krawczyk, D. C., Morrison, R. G., Viskontas, I., Holyoak, K. J., Chow, T. W., Mendez, M. F. , ... & Knowlton, B. J. ( 2008 ). Distraction during relational reasoning: The role of prefrontal cortex in interference control . Neuropsychologia , 46 (7), 2020–2032. Medline ,  Google Scholar
• Luft, J. A., & Roehrig, G. H. ( 2007 ). Capturing science teachers’ epistemological beliefs: The development of the teacher beliefs interview . Electronic Journal for Research in Science & Mathematics Education , 11 (2), 38–63. Google Scholar
• Matthewson, J. H. ( 1999 ). Visual-spatial thinking: An aspect of science overlooked by educators . Science Education , 83 (1), 33–54. Google Scholar
• Munby, H. ( 1984 ). A qualitative approach to the study of a teacher’s beliefs . Journal of Research in Science Teaching , 21 (1), 27–38. Google Scholar
• Mayer, R. E. ( 2019 ). Cognitive theory of multimedia learning . In Mayer, R. E. (Ed.), The Cambridge handbook of multimedia learning (pp. 43–71). Cambridge: Cambridge University Press. Google Scholar
• Mayer, R. E., & Fiorella, L. ( 2019 ). Principles for reducing extraneous processing in multimedia learning: Coherence, signaling, redundancy, spatial contiguity, and temporal contiguity principles . In Mayer, R. E. (Ed.), The Cambridge handbook of multimedia learning (pp. 279–315). Cambridge: Cambridge University Press. Google Scholar
• Miller, K. R., & Levine, J. S. ( 2011 ). Prentice Hall Biology: California . Upper Saddle River, NJ: Pearson Prentice Hall. Google Scholar
• Miyake, A., Friedman, N. P., Emerson, M. J., Witzki, A. H., Howerter, A., & Wager, T. D. ( 2000 ). The unity and diversity of executive functions and their contributions to complex “frontal lobe” tasks: A latent variable analysis . Cognitive Psychology , 41 (1), 49–100. Medline ,  Google Scholar
• Muijs, D., & Reynolds, D. ( 2015 ). Teachers’ beliefs and behaviors: What really matters? Journal of Classroom Interaction , 50 (1), 25–40. Google Scholar
• National Academy of Sciences . ( 2000 ) How people learn: Brain, mind, experience, and school . Washington DC. Google Scholar
• National Research Council . ( 2012 ). A framework for K–12 science education practices, crosscutting concepts, and core ideas . Washington, DC: National Academies Press. Google Scholar
• Next Generation Science Standards Lead States . ( 2013 ). Next Generation Science Standards: For states, by states . Washington, DC: National Academies Press. Google Scholar
• Padilla, M. J. ( 2007 ). Focus on life science: California . Upper Saddle River, NJ: Pearson Prentice Hall. Google Scholar
• Phillips, L. M., Norris, S. P., & Macnab, J. S. ( 2010 ). Visualization in mathematics, reading and science education (Vol. 5 ). New York, NY: Springer Science & Business Media. Google Scholar
• Richland, L. E., & McDonough, I. ( 2010 ). Learning by analogy: Discriminating between potential analogies . Journal of Contemporary Educational Psychology , 35 (1), 28–43. Google Scholar
• Roediger, H.L., & Karpicke, J. D. ( 2006 ). Test-enhanced learning taking memory tests improves long-term retention . Psychological Science , 17 (3), 249–255. Medline ,  Google Scholar
• Roth, K. J., Druker, S. L., Garnier, M. L., Lemmens, M., Chen, C., Kawanaka, T. , ... & Gonzales, P. ( 2006 ). Teaching science in five countries: Results from the TIMSS 1999 video study (NCES 2006-0110) . Washington, DC: National Center for Education Statistics. Google Scholar
• Schnotz, W. ( 2019 ). Integrated model of text and picture comprehension . In Mayer, R. E. (Ed.), The Cambridge handbook of multimedia learning (pp. 72–103). Cambridge: Cambridge University Press. Google Scholar
• Schnotz, W., & Bannert, M. ( 2003 ). Construction and interference in learning from multiple representation . Learning and Instruction , 13 , 141–156. Google Scholar
• Simms, N. K., Frausel, R. R., & Richland, L. E. ( 2018 ). Working memory predicts children’s analogical reasoning . Journal of Experimental Child Psychology , 166 , 160–177. Medline ,  Google Scholar
• Smith, M. S., Hughes, E. K., Engle, R. A., & Stein, M. K. ( 2009 ). Orchestrating discussions . Mathematics Teaching in the Middle School , 14 (9), 548–556. Google Scholar
• Stein, M. K., Engle, R. A., Smith, M. S., & Hughes, E. K. ( 2008 ). Orchestrating productive mathematical discussions: Five practices for helping teachers move beyond show and tell . Mathematical Thinking and Learning , 10 (4), 313–340. Google Scholar
• Sweller, J., van Merrienboer, J. G., & Paas, G. W. C. ( 1998 ). Cognitive architecture and instructional design . Educational Psychology Review , 10 , 251–296. Google Scholar
• Wallace, C. S. ( 2014 ). The role of science teachers’ beliefs in international classrooms . In Evans, R.Luft, J.Czerniak, C.Pea, C. Overview of the Role of teacher Beliefs in Science Education (pp. 17–31). Rotterdam: Sense Publishers. https://doi.org/10.1007/978-94-6209-557-1 Google Scholar
• Waltz, J. A., Lau, A., Gewal, S. K., & Holyoak, K. J. ( 2000 ). The role of working memory in analogical mapping . Memory & Cognition , 28 (7), 1205–1212. Medline ,  Google Scholar
• Zook, K. B. ( 1991 ). Effects of analogical processes on learning and misrepresentation . Educational Psychology Review , 3 (1), 41–72. Google Scholar
• Designing Exhibits to Support Relational Learning in a Science Museum 26 March 2021 | Frontiers in Psychology, Vol. 12

Submitted: 19 November 2019 Revised: 14 September 2020 Accepted: 29 September 2020

© 2020 J. Hansen and L. Richland. CBE—Life Sciences Education © 2020 The American Society for Cell Biology. This article is distributed by The American Society for Cell Biology under license from the author(s). It is available to the public under an Attribution–Noncommercial–Share Alike 3.0 Unported Creative Commons License (http://creativecommons.org/licenses/by-nc-sa/3.0).

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

• View all journals
• Explore content
• About the journal
• Publish with us
• Sign up for alerts
• Open access
• Published: 28 June 2024

Immersive scene representation in human visual cortex with ultra-wide-angle neuroimaging

• Jeongho Park   ORCID: orcid.org/0000-0003-4260-3435 1 ,
• Edward Soucy 2 ,
• Jennifer Segawa 2 ,
• Ross Mair   ORCID: orcid.org/0009-0004-8085-407X 2 , 3 , 4 &
• Talia Konkle   ORCID: orcid.org/0000-0003-1738-4744 1 , 2 , 5  

Nature Communications volume  15 , Article number:  5477 ( 2024 ) Cite this article

Metrics details

• Object vision

While human vision spans 220°, traditional functional MRI setups display images only up to central 10-15°. Thus, it remains unknown how the brain represents a scene perceived across the full visual field. Here, we introduce a method for ultra-wide angle display and probe signatures of immersive scene representation. An unobstructed view of 175° is achieved by bouncing the projected image off angled-mirrors onto a custom-built curved screen. To avoid perceptual distortion, scenes are created with wide field-of-view from custom virtual environments. We find that immersive scene representation drives medial cortex with far-peripheral preferences, but shows minimal modulation in classic scene regions. Further, scene and face-selective regions maintain their content preferences even with extreme far-periphery stimulation, highlighting that not all far-peripheral information is automatically integrated into scene regions computations. This work provides clarifying evidence on content vs. peripheral preferences in scene representation and opens new avenues to research immersive vision.

Introduction

When we look at the world, we feel immersed in a broader visual environment. For example, the experience of a view of an expansive vista from the top of a mountain is not the same as when looking at a picture of the same view. One key difference is that in the real world, we sense a >180 degrees view of the environment at each glance. Indeed, while our fovea and macula ensure high-resolution input at the center of gaze, there is an equally impressive expanse of peripheral vision: with 170 degrees sensed by a single eye, and up to 220 degrees of the extreme far-periphery sensed by the two eyes combined 1 . What are the neural processes by which this immersive visual experience of the broader environment is constructed in the human visual system?

Seminal research has identified three brain regions in the human brain that show a clear role in high-level visual scene perception 2 , 3 . There are parahippocampal place area (PPA 4 ) in the temporo-occipital cortex, retrosplenial cortex (RSC 5 ) or medial place area (MPA 6 ) in the medial side along the parietal-occipital sulcus, and occipital place area (OPA 7 , 8 ) in the parieto-occipital cortex. Extensive neuroimaging studies have characterized tuning properties of these regions and their complementary roles in scene perception, regarding recognition 9 , 10 , 11 , 12 , 13 and navigation 14 , 15 , 16 , 17 , 18 , 19 , 20 in particular.

However, the constraints of standard fMRI image projection setup have limited scene perception research to the central 10-20 degrees of the visual field, with scene properties inferred from postcard-like picture perception. Thus, it remains unknown how a scene activates the visual system when it is presented across the full visual field, providing a more immersive first-person view. Would this alter the way we define the scene regions along the cortical surface (e.g., a larger cortical extent, or new scene regions)? More generally, what are the neural processes that construct a visual scene representation when far-peripheral information is available?

Here, drawing inspiration from an infant fMRI study 21 , we introduce an innovative image projection setup, which enables the presentation of ultra-wide-angle visual stimuli in an fMRI scanner. In typical scanning setups, stimuli are presented to humans lying supine in the scanner by projecting onto a screen outside of the scanner bore, while the participants look out through a head coil at a small mirror reflecting the screen behind them. With this setup, the maximum visual angle of a projected image is ~15–20 degrees. We modified this setup, by bouncing the projected image off two angled mirrors, directly onto a large, curved screen inside the scanner bore. This allowed us to project images about 175 degrees wide, stimulating almost the entire visual field.

While there have been prior approaches to establish wide-angle presentation, they were mainly centered on studying retinotopic properties in early visual areas, presenting expanding rings and rotating wedges in black and white 22 , 23 , 24 , 25 . Thus, for testing high-level visual areas, which involves presenting more complex images (e.g., faces or scenes), different solutions imposed specific limitations. For example, one approach enabled researchers to project images up to 120 degrees, but only to one eye at a time, and onto a screen that was 3 cm from an eye, requiring participants to view stimuli with a custom contact lens 24 , 25 , 26 , 27 , 28 , 29 . More recently, a high-resolution MR-compatible head mounted display was developed, but the maximum field-of-view is ~52 degrees wide (Nordic Neuro Lab). Our solution was developed with the intention of studying high-level visual perception by providing as expansive and natural visual experience as possible. Further, our approach does not require participants to wear additional devices, and leverages a relatively low-tech solution that can be implemented in other scanning facilities.

With this full-field neuroimaging setup, we first chart the cortex with far-peripheral sensitivity. Then, we leverage this wide-angle setup to entertain questions about what it means to be a scene and the implications for the responses of classic scene-selective regions. For example, perhaps any image content presented in the far-periphery is part of a scene, and should be automatically integrated into the computations of high-level scene regions. From an embodied, ego-centric perspective, this is a reasonable account. Alternatively, perhaps the scene regions are more like high-level pattern analyzers that are sensitive to particular kinds of image statistics (e.g., open/closed spatial layout, contour junctions, etc.) rather than to the retinotopic location of the visual stimulation per se. Indeed, in the scene perception literature, there is evidence for both accounts. The neuroimaging studies with 0–20 degrees of the visual field showed that the classic scene regions are modulated both by the scene content (over other semantic category contents like faces) and by peripheral stimulation 6 , 7 , 30 , 31 , 32 . We now extend the scope of this investigation to the entire visual field and revisit this question.

Ultra-wide-angle fMRI

To accomplish ultra-wide-angle visual presentation in the scanning environment, we installed two angled mirrors near the projector such that the projected image was cast directly into the scanner bore, onto a custom-built curved screen positioned around a person’s head (Fig.  1 , Supplementary Fig.  3) . Additionally, given the visual obstruction of the top of the head coil, we simply removed it, allowing participants to have an unobstructed view of the curved screen. Through signal quality check protocols, we confirmed that the lack of top head coil did not have critical impacts on MRI signals for occipital and parietal cortices (see Supplementary Fig.  1 for more details).

An image is bounced off two angled mirrors and directly projected onto a curved screen inside the scanner bore.

To compensate for the curved screen, we developed code to computationally warp any image, to account for the screen curvature and tilted projection angle (Fig.  2 ). Given the geometrical constraints of our MRI room, only a subset of pixels could be projected onto the screen, resulting in substantially lower image resolution compared to other standard projection systems, particularly in the vertical dimension (see Methods).

A rectangular image (1024 × 768 pixels) was computationally warped to match the size and curvature of the tilted screen. Due to the geometrical constraints of the room, only a subset of pixels could be projected onto the screen (828 × 284 pixels). On the curved screen, the aspect ratio of the original image was maintained on the display surface.

Further, we found that when projecting natural scene images across the full field, using standard pictures taken from typical cameras lead to highly distorted perceptions of space—a picture with a compatible wide field-of-view was required. Thus, for the present studies, we built virtual 3D environments in Unity game engine (Unity Technologies, Version 2017.3.0), where we could control the viewport height and field-of-view when rendering scene images. Further details about the full-field fMRI setup can be found in the Methods and on our website ( https://jpark203.github.io/fullfield-neuroimaging ). Taken together, our solution enabled us to present images over 175 degrees, providing natural and immersive viewing experience.

Full-field eccentricity map

In Experiment 1, we first attempted to map the full visual field and chart an extended eccentricity map along the visual cortex. We used a classic retinotopic mapping protocol, where participants performed a fixation dot color detection task. Flashing checkerboards were presented in rings at five levels of eccentricity: (1) a center circle of 1.8 degrees radius, and (2) the inner and outer rings of 2.0–5.6 degrees, (3) 6.3–16.5 degrees, (4) 18.5–50.3 degrees, and (5) >55.3 degrees radius. The two farthest eccentricities were not possible with typical scanning setups, allowing us to stimulate cortical territory that has been inaccessible via direct visual input.

The cortical map of eccentricity preferences is shown in Fig.  3 . For each voxel, we compared responses to different eccentricity conditions, and colored the voxel based on the condition with the highest activation (hue). The resulting map revealed a systematic progression of preference from the center to far-periphery, covering an expansive cortical territory along the medial surface of the occipital lobe. In particular, we mapped strong responses to far-peripheral stimulation near the parieto-occipital sulcus (POS), extending beyond our typical eccentricity band maps (black dotted line, Fig.  3) . These results validate the technical feasibility of our ultra-wide-angle projection method, and to our knowledge, show the full-field mapping of eccentricity in the human brain that exceeds the scope of prior studies.

An example participant’s right occipital cortex is shown from a medial view. Each voxel is colored based on its preference for one of five eccentricity rings (right). In the group data, the black dotted line shows where a typical eccentricity map would end, and the black arrows show how much more the cortex can be stimulated with full-field neuroimaging. Individual brain maps from nine participants also show a consistent pattern of results. POS parieto-occipital sulcus.

Full-field scene perception

With this full-field neuroimaging set up, we next measured visual system responses to ultra-wide-angle, immersive real-world scenes and compared them to responses from visually-smaller postcard scenes and unstructured image-statistical counterparts.

Specifically, we created four different stimulus conditions that varied in presentation size (full-field vs. postcard), and content (intact vs. phase-scrambled scenes). The full-field images filled up the entire screen (175 deg wide), and the postcard images were presented at the center of screen in a much smaller size (though still 44 deg wide). The chosen size of postcard images was bigger than the maximum size in typical fMRI setups due to limited image resolution. We discuss this limitation further in the Discussion.

To match the image content across presentation sizes, the postcard images were rescaled from the entire full-field images, instead of cropping the center only. To vary the image content, the same scenes were phase-scrambled, preserving the summed spatial frequency energy across the whole image but disrupting all second-order and higher-level image statistics present in scenes 33 , 34 . Additionally, we also included a postcard-face condition where a single face was presented at the center of screen, in a similar visual size to the postcard-scenes. Each stimulus condition was presented in a standard blocked design (12 sec), and participants performed a one-back repetition detection task (see Methods for further details).

First, we asked how the visual cortex responds to full-field size images with intact scene content, compared to images with phase-scrambled scene statistics (Fig.  4 a). This contrast is matched in full-field retinotopic footprint, but different in the image content. Will the immersive full-field scenes recruit additional brain regions, e.g., with more extensive scene regions (in terms of cortical surface area), or separate brain areas away from the classic scene regions that were not found with the traditional fMRI setups due to the limited stimulus size?

The group data is shown on an example subject brain. Zoom-in views at each row are captured around the classic scene regions. a Image content contrast. A large portion of high-level visual areas, including the scene regions, shows higher activation for the intact scenes compared to the phase-scrambled scenes. b Visual size contrast. A large swath of cortex near the parieto-occipital sulcus is strongly activated when viewing a full-field scene compared to a postcard scene. PPA parahippocampal place area, RSC retrosplenial cortex, OPA occipital place area, POS parieto-occipital sulcus.

The whole-brain contrast map is shown with the group data in Fig.  4 a (Supplementary Fig.  4 for individual participants). We found qualitatively higher responses for intact scenes over the scrambled scenes along the ventral medial cortex, as well as dorsal occipito-parietal cortex. For comparison, we defined three scene ROIs by contrasting the postcard-scene vs. postcard-face condition, reflecting a more typical (non-full field) definition of these regions. Fig.  4 a shows the overlaid outlines of these classically-defined ROIs (PPA; OPA; RSC). Note that these ROIs reflect group-level ROIs for visualization, but all ROIs were defined in individual subjects in independent data. Qualitative inspection reveals that these ROIs largely encircle the strongest areas of scene-vs-scrambled response preferences. In other words, it is not the case that the full-field stimulation leads to strong scene content-preferring responses that clearly extend well beyond the postcard-defined ROI boundaries.

One important note is that our postcard-sized stimulus was still rather large (44 degrees) relative to the visual size presented in typical set ups (15-20 degrees). Thus, the present data indicate only that the extent of activated cortical surface is not much increased by a relatively dramatic stimulus size increase from 44 to 175 deg. If there is increasing cortical scene-selective territory as a function of visual angle, it is limited to visual size increases from 15-44 degrees. More detailed parametric visual size mapping is required to answer this question. For the purposes of the present work, these results reveal that the standard contrasts for defining classic scene regions reflect stable functionally defined regions, across both our postcard and full-field presentation sizes.

Next, we asked how the visual cortex responds to full-field scenes compared to postcard scenes. This contrast is matched in content (i.e., identical scene images that have been rescaled), but different in retinotopic footprint (Fig.  4 b). This allows us to examine which cortical territory is more active under an immersive visual experience of a scene view, compared to postcard scene perception.

A whole-brain contrast map is shown in Fig.  4 b (Supplementary Fig.  5 for individual participants). This map shows that cortex near the POS is activated significantly more to full-field scenes than postcard scenes. This cortex showed far-peripheral visual field preference in Experiment 1, and effectively corresponds to the far-peripheral parts of early visual areas. Thus, it is likely that this cortex is not uniquely attributed to scene content presentation per se, but to any far-peripheral visual stimulation (which we explore further in the next experiments). Anatomically, this swath of cortex is largely adjacent to and mostly non-overlapping with classic scene regions, PPA and OPA, and anterior part of RSC. Thus, while it could have been that the full-field vs. postcard contrast would strongly encompass the scene-selective regions, this was not the case.

Effects of visual size and scene content

The whole-brain contrasts did not show clear evidence for a new scene region, or more extensively activated cortical surface area from the classic scene regions. Thus, we focused our quantitative analyses on these classic scene ROIs defined at the postcard visual size, and explored the extent to which each scene region is modulated by the visual size and scene content.

In addition to the scene ROIs, we defined a “Peripheral-POS” (parietal-occipital sulcus) region, using the retinotopy protocol data from Experiment 1. Specifically, we selected voxels that survived a conjunction contrast between pairs of the far-peripheral eccentricity ring condition and all other eccentricity ring conditions. Further, we removed the small proportion of voxels in the Peripheral-POS which spatially overlapped with independently defined RSC (mean = 5.6%, std = 7.7% of Peripheral-POS voxels).

The results of the ROI analyses are shown in Fig.  5 . Broadly, this 2 × 2 design reveals a powerful transition in the invariances of the responses, from cortex with retinotopic selectivities to scene content selectivities. Specifically, the Peripheral-POS region showed clear retinotopic modulation: there was a large effect of full-field vs. postcard sizes (F(1, 36) = 518.6, p  < 0.01, etaSq = 0.91), with only weak effect of image content (F(1, 36) = 11.7, p  < 0.01, etaSq = 0.02), and no interaction between these factors (F(1, 36) = 1.8, p  = 0.2). Put succinctly, this region shows clear retinotopic modulation, with little sensitivity to higher-order scene image content.

The anatomical locations of each ROI are illustrated on a schematic brain map in the middle (top: medial side, bottom: ventral surface of the right hemisphere). Each ROI panel shows the mean beta averaged across participants ( n  = 10) for each condition. Individual data are overlaid on top of the bars as dots. The main effect of visual size (blue vs. purple) and the main effect of content (dark vs. light) were significant in all ROIs. The significant interaction was found only in the PPA and RSC. The FFA result is in Supplement Fig.  6 . Post Postcard, PostScr Postcard Scrambled, FF full-field scenes, FFscr full-field scenes scrambled, PPA parahippocampal place area, RSC retrosplenial cortex, OPA occipital place area, FFA fusiform face area, POS parieto-occipital sulcus.

In contrast, both the PPA and the OPA showed the opposite pattern. That is, there were large effects of scene content vs. scrambled content (PPA: F(1, 36) = 535.2, p  < 0.01, etaSq = 0.86; OPA: F(1, 36) = 168.9, p  < 0.01, etaSq = 0.8), with only small effects of image size (PPA: F(1, 36) = 44.7, p  < 0.01, etaSq = 0.07; OPA: F(1, 36) = 5.1, p  < 0.05, etaSq = 0.02). There was a very small interaction of these factors in PPA, but not in OPA, with slightly higher activation in PPA for scenes in full-field presentation (PPA: F(1, 36) = 6.5, p  < 0.05, etaSq = 0.01; OPA: F(1, 36) = 0.6, n.s.). Thus, intact scenes drive much higher response than the phase-scrambled scenes in PPA and OPA, generally independently of the presentation size (darker vs. lighter color bars, Fig.  5) .

The RSC immediately abuts the Peripheral-POS region. Interestingly, it has a slightly more intermediate pattern, though it is more like the other high-level scene regions. That is, RSC showed a large effect of scene content (RSC: F(1, 32) = 141.1, p  < 0.01, etaSq = 0.52) and a moderate effect of visual size (RSC: F(1, 32) = 93.1, p  < 0.01, etaSq = 0.34), with only very weak interaction between them (RSC: F(1, 32) = 4.3, p  < 0.05, etaSq = 0.02). Taken together, these data reveal a clear pattern: classic scene regions have strong overall responses for image content, which is maintained over dramatically different visual sizes and a qualitatively different immersive experience, with relatively weaker modulation by the visual size of stimulus.

As a control, we also examined responses in the face-selective FFA (Supplementary Fig.  6) . While the overall responses to all four conditions were quite low, there was a small but statistically reliable main effect of visual size, with higher overall activation in full-field over postcard views (F(1, 36) = 8.9, p  < 0.01, etqSq = 0.19). The responses of this control region suggest that full-field stimulation might partly provide a more general boost to the visual system (e.g., via arousal). On this account, the scene regions’ slight preference for full-field stimulation might reflect a more general drive, further amplifying the dissociation between tuning for content and peripheral stimulation.

Thus, from the far-peripheral retinotopic cortex to the classic scene regions, there is a relatively abrupt transition in tuning along the cortical sheet. The far-peripheral retinotopic cortex shows only weak content differences. Adjacent scene-selective cortex amplifies these scene vs. scrambled content differences, regardless of whether or not the content stimulates the far periphery.

Far-peripheral stimulation without the central visual field

The previous experiment showed that scene regions are modulated dominantly by the image content, much less so by the visual size. However, postcard and full-field scenes both stimulate the central 45 degrees of the visual field. Thus, it is possible that the scene content preferences we observed are actually primarily due to central visual field stimulation. Are these scene content preferences also evident when only stimulating the far-periphery? In Experiment 3, we asked how far in eccentricity this scene preference is maintained.

We also asked the parallel question for face-selective regions. FFA is traditionally defined by contrasting responses to face vs. object image content presented in the center of the visual field. What happens when faces are presented in the far-periphery? Do face-selective regions also maintain their face content preferences when only presenting the content in the very far-peripheral visual field? Or, will any structured image content be represented increasingly more like a “scene” and drive scene regions, as it is presented farther from the center?

To directly test these questions, we generated a new stimulus set, depicting different content across the visual field, with increasing degrees of central “scotoma” 35 , that have matched retinotopic footprint to full-field scenes but differ in their content (Fig.  6 ). As in the previous experiment, we included both wide-angle rendered 3D scenes and their phase-scrambled counterparts. As a proxy for “full-field faces”, we made face arrays, in which multiple individual faces were presented throughout the full visual field. To avoid crowding effect and make each face recognizable (at basic category level), we adjusted the size of faces as a function of eccentricity (see Methods). Object arrays were generated in the same manner with individual small objects.

To stimulate only the peripheral visual field, we removed the central portion of the image by creating an artificial scotoma that systematically varied in size. There were five levels of scotomas including the no-scotoma condition (columns). We filled in the remaining space with four different kinds of image content: intact scenes, phase-scrambled scenes, object array, and face arrays (rows). For the object and face arrays, the size of individual items was adjusted to account for cortical magnification. *For copyright reasons, human faces have been substituted with illustrations in this manuscript, and objects were substituted with example images without copyright.

Then, we parametrically masked the central portion of images at 5 sizes (0, 30, 58, 88, and 138 degrees in diameter; see Fig.  6) . We measured brain responses to these 20 conditions, using a blocked design (see Methods). Participants were asked to perform a one-back repetition detection task while fixating their eyes at the center of screen. As before, we defined the classic scene ROIs using the same method (i.e., postcard-scene vs. postcard-face) from independent localizer runs.

We first examined responses of scene and face ROIs (Fig.  7 ). As expected, when there is no scotoma, all regions showed preferences for either scenes or faces relative to other categories. As the size of the central scotoma increases, leaving only increasingly peripheral stimulation, the results showed that content preferences across all ROIs were generally maintained. Through the penultimate scotoma condition (88 deg), all scene regions showed significantly higher activation for scenes compared to face arrays, object arrays, and phase-scrambled scenes (see Supplementary Tables for statistical test results).

In each panel, the line plot (error bands = standard error of the mean) shows how the response of each ROI changed as we increasingly removed the central visual field stimulation via scotoma, leaving only the peripheral stimulation. The call-out box with a bar plot (* p  < 0.05, two-sided paired t-test; see Supplement Tables for the full report of the statistical tests) shows responses for each image content at the largest scotoma condition (138 deg diameter). a , b Overall, PPA and RSC maintained their scene preference over faces across all scotoma conditions, whereas c the OPA maintained the preference until the penultimate condition. d The FFA also maintained its content preference for faces across all scotoma conditions. PPA parahippocampal place area, RSC retrosplenial cortex, OPA occipital place area, FFA fusiform face area.

The pattern at the farthest scotoma condition (138 deg) varied by the ROI and stimulus. RSC showed strong scene preference against all other image contents (Fig.  7 b, Supplementary Table.  2) . However, OPA’s scene preference did not hold at the 138 deg scotoma condition (Fig.  7 c, Supplementary Table.  3) . The PPA showed significantly higher activation for scenes compared to face arrays, but this activation level was not different from object arrays (t(9) = 2.2, n.s.; Figure  7 a; Supplementary Table.  1) . These results are also depicted on the cortical surface in Fig.  8 (Supplementary Fig.  7 for individual participants), showing the contrast of face vs. scene content, as the presentation is restricted increasingly peripherally. Overall, our results show that scene regions can be driven by content differences through a purely peripheral route, beyond at least 88 deg, that does not require central presentation.

This figure shows the whole-brain contrast between the scenes (red) and faces (blue), at each scotoma condition (columns). a Ventral view with PPA and FFA. b Medial view with RSC. c Lateral view with OPA. PPA parahippocampal place area, RSC retrosplenial cortex, OPA occipital place area, FFA fusiform face area.

Next we turned to FFA. If the presence of faces at the central visual field is necessary to drive FFA responses, then we would have expected the face preference to exist only in the no-scotoma or small scotoma conditions. However, that is not what we found. Instead, face-selective FFA shows the same pattern as the scene-selective regions. That is, FFA responded more to face content than other image content, across all scotoma levels, even at 138 degrees (see Supplementary Table.  4 for stats). This pattern of results is also evident in the cortical maps of Fig.  8 (Supplementary Fig.  7 for individual participants). Overall, these results clearly demonstrate that face-selectivity is present even when faces are presented in the very far periphery only. Thus, this result suggests that there is also a far-peripheral route to drive face-selective responses in the FFA, which does not require direct stimulation of the central visual field.

Finally, we wondered whether participants would actually be aware of the stimulus condition when it was presented in the far 138+ degrees of the visual field. To explore this, we conducted a brief categorization test during the anatomical scan. Either an object array or face array was presented with one of four scotoma sizes, and participants did a 2-alternative-forced-choice task. We found that participants were nearly perfect through the penultimate scotoma condition (30 deg: mean = 0.98, s.e = 0.02; 58 deg: mean = 0.96, s.e. = 0.03; 88 deg: mean = 0.99, s.e. = 0.01). The accuracy at the farthest eccentricity was more variable, but still statistically above chance (mean = 0.64, s.e. = 0.04; t(11) = 4.0, p  < 0.01). We note that only a limited number of trials were conducted due to time constraints, so these results should be interpreted with caution. However, the current results suggest that participants, on average, were weakly able to do the basic-level categorization, with only extreme peripheral visual information present.

Peripheral bias in scene regions

Lastly, in the classic scene regions, we found only minimally higher activation for full-field scenes relative to postcard scenes. Is this finding at odds with previously reported “peripheral bias”? Previous studies indicating a peripheral bias have shown increased activation in the PPA when the stimulated location moves from the central visual field to the periphery, up to 20 deg in diameter 30 , 36 . Two points are worth clarifying. First, our comparison between full-field scenes vs. postcard scenes is not actually a direct test of central vs. peripheral tuning, as both of these conditions stimulate the central visual field. Second, how much a region is activated depends on its receptive field (RF) size and location. So, for example, if a region’s RF completely encompasses the 44 deg diameter center of the visual field (i.e., postcard scene presentation size), that means this brain region’s RF would be stimulated in both postcard and full-field scenes, predicting not much activation difference.

We thus ran an exploratory analysis that examined each ROI’s response to the increasing eccentricity ring checkerboards used in Experiment 1. A peripheral bias account would intuitively predict that increasing peripheral stimulation would lead to a corresponding activation increase in each of these scene regions. However, that is not what we found. Instead, each scene ROI showed a different pattern in response to these eccentricity rings (Fig.  9 ).

a PPA response increases until the penultimate condition then drops at the extreme periphery. b RSC response was rather flat then jumped after the third ring, clearly showing its preference for the far-periphery. c OPA showed a mild peak around the third ring. d FFA showed the opposite pattern to a – c , demonstrating its preference for the central visual field. PPA parahippocampal place area, RSC retrosplenial cortex, OPA occipital place area, FFA fusiform face area.

The PPA had increasing activation with increasingly peripheral eccentricity rings (up to 37–100.6 deg diameter) but dropped at the farthest, most peripheral stimulation condition (>110 degrees). The OPA was similar to PPA, but with a nominal peak activation at the 3rd level of eccentricity (12.6–33 deg). Finally, RSC’s activation to central checkerboards was not significantly different from baseline for the first three levels, and then abruptly increased for both the two most extreme peripheral rings. Thus, neither PPA nor OPA showed strong sensitivity to ultra-peripheral generic stimulation (flashing checkerboard), showing a limit on the general peripheral bias hypothesis of scene regions.

Are these ROI responses across levels of eccentricity consistent with the visual size effects between full-field and postcard conditions? The size of the postcard scene (44 deg diameter) is most similar to the size of the inner circle at the fourth eccentricity ring (37 deg). So, in a rudimentary way, the stimulated visual field by both the last two eccentricity rings (>37 deg) roughly corresponds to the additionally stimulated visual field by the full-field scenes compared to the postcard scenes (>44 deg). Both PPA and OPA have stronger responses for the first three levels of eccentricity than the final two levels, and consistently showed little additional response to full-field scenes relative to postcard scenes. Meanwhile, RSC shows weaker responses for the first three levels of eccentricity, and more for the most peripheral conditions; and consistently, RSC showed stronger responses for full-field conditions regardless of content. Thus, the activation differences over eccentricity rings are indeed consistent with the visual size modulation effect of each scene region, observed in Experiment 2.

In sum, this post-hoc analysis is consistent with the previously known notion that peripheral bias—peripheral stimulation activates the scene regions more than the foveal stimulation. However, our results also place updated constraints on this account. The peripheral bias in the scene regions is present only up to a certain eccentricity, and this differs depending on each scene region. We offer that the thinking of a general peripheral bias is thus not appropriate, and the responsiveness over the visual field might be better understood in the context of RFs. Future work employing population RF mapping can be used to further clarify and chart the far-peripheral RF structure across these cortical regions.

In this study, we established a method to present ultra-wide-angle visual stimuli in the scanning environment. With this new tool, we were able to measure neural responses to the extreme far-periphery and chart the ultra-wide eccentricity map in the human brain beyond the scope of prior studies. We then examined the neural basis of full-field scene perception. We found that classic scene regions are tuned to scene content that is robust to changes in the visual size of scenes, suggesting a sharp tuning transition from adjacent far-peripheral retinotopic cortex to scene content regions. We also found scene and face-selective regions maintained their content preferences even in conditions of extreme peripheral stimulation, highlighting the existence of a far-peripheral route that has yet to be fully investigated. Finally, only RSC showed systematically higher responses at the farthest eccentricity, where both PPA and OPA had weaker responses, clarifying new limits on the peripheral bias of scene regions. Broadly, this work brings unique empirical evidence to clarify debates about the issues of content and peripheral preferences in scene representation and introduces an innovative method for investigating more naturalistic, immersive scene perception inside a scanner.

The full-field neuroimaging method allowed us to gain some new insights into the classic scene regions. First, we gained a better understanding of what it means to be a scene. While it has been well established that PPA, RSC, and OPA are scene-selective regions, the definition of a scene has been used in a somewhat mixed way. On one hand, a scene can be a set of visual patterns with particular kinds of higher-order image statistics. On the other hand, anything (including objects or faces) that falls in the far periphery can be part of a scene. This account is motivated by intuitions that part of what it means to be a scene is to have content that extends beyond the view. Leveraging the ultra-wide-angle image projection, our study directly compared these two accounts.

Overall results are clearly in favor of the first hypothesis. That is, not just any information in the far-periphery becomes a scene and is automatically integrated into the computations of scene regions. Even when faces or objects are at the far periphery, they do not drive the scene regions more than they would normally do at the central visual field. Instead, the classic scene regions are tuned to particular higher-order image statistics that are distinctive from visual features of other semantic categories, although there are some further differences among the scene regions 2 , 3 . This makes sense: many of the core visual features important for the scene regions are not much disrupted by the visual size or location change. For example, spatial layout 34 , 37 , statistics of contour junctions 38 , surface properties like material or texture 12 , 39 , or objects present in a scene 40 can be extracted similarly in both postcard and full-field scenes. However, it is also worth emphasizing that while these features do not have to be present in specific retinotopic locations, in real visual experience, useful visual cues for those (e.g., walls, planes, or boundaries) tend to be at the periphery rather than the center, providing an ecological explanation why the scene regions are developed to have sensitivity to visual information at the periphery.

Additionally, the access to the far-periphery provided a new perspective on the anatomical locations of the scene regions. We showed that three scene regions are very closely positioned to the far-peripheral cortex along the POS. When we perceive a full-field view, this medial brain region and the classic scene regions are activated together, forming a large ring-shaped portion of the cortex along the POS. In other words, the classic scene regions might be connected together by the far-periphery preferring cortex. This observation allows us to realize that the scene regions are actually proximal to each other anatomically. This intuition is not easily captured from the typical flattened brain map, because the cut is made along the fundus of calcarine sulcus 41 , splitting the retinotopic map into upper and lower visual quadrants, which in turn places PPA and OPA on opposite sides of this flat map (e.g., see ref. 42 for how object-selective cortex is organized between these PPA and OPA regions). Our schematic map of the medial surface (Fig.  10 ), in contrast, keeps the visual field maps intact, emphasizing the proximity between the scene regions and their relationship to the retinotopic map.

The scale and shape of retinotopic map is not accurately presented as the actual data. Instead, this flattened map of the medial view emphasizes the idea that the three scene regions might be connected via the far-peripheral cortex. VF visual field, PPA parahippocampal place area, RSC retrosplenial cortex, OPA occipital place area.

This view naturally lends an explanation why the PPA has upper visual field bias, OPA has lower visual field bias, and RSC does not show clear bias to either upper or lower visual field 31 . Further, this large-scale cortical organization may be related to recently proposed place-memory areas that are positioned immediately anterior to each of the scene-perception areas 43 . In particular, the organization is suggestive of a hierarchical representational motif, with systematic transformations of representation from retinotopic far-peripheral cortex to perceptual scene structure of the current view to more abstract scene memory.

Another interesting question is the relationship between RSC and area prostriata, which is located in the fundus of the calcarine sulcus, just anterior to the far-peripheral V1 29 . The prostriata has a distinctive representation from V1 over a wide visual field (up to 60 deg), and responds more to fast motion (570 deg per sec) than the moderate-speed motion (38 deg per sec) 29 . Moving dorsally along the POS, there is also human V6 that has the sensitivity to coherent field motion (e.g., spiral motion vs. random dots) or optic flow when compatible with self-motion 44 , 45 . While it requires further investigation whether the prostriata overlaps with the functionally defined RSC, it is possible that its sensitivity to peripheral motion might be used for representing dynamic scenes, to support spatial navigation.

The scene regions and even the fusiform face area both showed their content preference at the extreme far-periphery. How do these regions process stimuli at the far periphery?

Many studies have shown that face-selective regions respond more strongly to foveal stimulation, whereas scene-selective regions respond more strongly to peripheral stimulation 30 , 36 , 46 . Further, stronger functional connectivity was found between foveal V1 and face-selective regions (and between peripheral V1 and scene-selective regions), in human adults 47 , as well as in human and macaques neonates 48 , 49 . More recent study using diffusion MRI also showed higher proportion of white matter connection between foveal early visual cortex and ventral face regions (e.g., fusiform face area; FFA) 50 . Together, these results imply eccentricity-based preferential connection between early visual cortex and higher category-selective regions, which does not easily explain our findings.

One possibility is that there are meaningful connections across all eccentricities between the early visual cortex and the higher visual areas, even though some connections to a particular eccentricity are more weighted (e.g., FFA and foveal V1). Then, FFA might still show a somewhat weaker but preferential response to faces at the far periphery, as long as the stimuli are presented with appropriate visual size and arrangement to accommodate cortical magnification and crowding.

Another possibility is that attention temporarily adjusts RF properties of high-level visual areas. A study using the population receptive field (pRF) method showed that the pRFs of FFA were located more peripherally and larger during a face task (one-back judgment) than during a digit judgment task, resulting in extended coverage of the peripheral visual field 51 . While there was no control task condition in our experiments, the one-back repetition detection task could have helped incorporate far-peripheral stimuli into computations.

Additionally, there might be other input connections to the high-level visual areas outside the ventral pathway, perhaps via a subcortical route (e.g., superior colliculus) 52 , 53 or from the lateral surface. For example, the diffusion MRI study showed that lateral face regions (e.g., posterior STS-faces) have uniformly distributed connections across overall early visual cortex eccentricities, in contrast to the foveal-biased ventral FFA 50 . This suggests that the processing of faces is not limited to the central visual field, as they can also be processed at the periphery, especially in dynamic or social situations 54 , 55 . It is possible that the peripheral face selectivity observed in FFA may reflect responses from those lateral face areas. Further investigation is necessary to better understand these peripheral routes and how they support the transition from eccentricity-based to content tuning.

Lastly, another possibility to consider is that this effect is driven by non-compliant subjects who moved their eyes to the periphery. However, if participants always shifted their gaze towards the periphery, activation levels at the largest scotoma would match those in the no-scotoma condition, and if such eye movements happened only occassionally, it would likely result in greater variance in the far-periphery condition, which we did not observe. Further, moving your eyes beyond 70 degrees requires considerable effort and some discomfort. Thus we think this account of the responses is unlikely. While the setup here precludes the use of traditional eye-tracking equipment, emerging computational eye-tracking methods that extract eye gaze from the EPI images could prove a valuable complement to this method in future studies 56 .

Achieving a wide-angle (>15–20 deg) visual stimulation in an fMRI scanner has been a goal since the early days of fMRI 57 , 58 . For example, in the early 2000s, researchers were able to stimulate wider visual field up to 100 deg, mapping retinotopic area V6 in humans 22 . To achieve this, a relatively large flat screen (260 × 185 mm) was positioned inside the scanner bore, and the closer distance between the screen and the eyes allowed it to stimulate a larger portion of the visual field. However, this screen size was too large to be easily adaptable to other MRI scanners or conventional head coils. Another research group achieved 100 deg wide stimulation with a smaller screen (140 mm wide), but they used the combination of glasses and prism to enlarge the size of projected stimuli 23 , 59 .

In the next decades, the angle of image projection was pushed up to 120 deg wide 24 , 25 , 26 , 27 , 28 , 29 . These approaches leveraged monocular viewing–presenting the image to only one eye. In these setups, a small screen was positioned very close to the eyes (3 cm), and participants had to wear a contact lens to get help with focus and fixation at such a short distance. And, most recently, stimulation of the entire visual field was achieved 60 . Using custom-built goggles with white light-emitting diodes (LEDs), they were able to functionally localize the temporal monocular crescent, which requires wide-angle projection beyond 120 deg.

While much of this early work focused on retinotopy, our aim was to develop an approach that does not require participants to wear any devices and allows them to see stimuli as naturally as possible as they do outside the scanner. And, we focus here on exploring the perception and representation of high-level visual information presented extensively across the visual field. An advantage of our approach is that apparatus can be built at relatively low cost. We used a pair of mirrors to control the image projection trajectory, and the curved screen can be assembled with 3D-printed plastic parts. We share the design files and all specifications via a public website and community mailing list to support ultra-wide-angle neuroimaging ( https://jpark203.github.io/fullfield-neuroimaging ).

One of the current challenges of our ultra-wide-angle projection setup is that we are scanning without the top head coil because it blocks the peripheral view. While the data quality was still viable, there was a clear decrease of tSNR in all of the main ROIs (Supplementary Fig.  2) . The lack of top head coil could also limit the scope of research topics, especially if they involve investigating on the frontal lobe. Another main challenge is a limited image resolution (2–4 pixels/degree). Due to physical constraints of the scanner room, only ~30% of pixels from the projected image could be on the screen. This is because as the distance between the projector and the screen (inside the scanner bore) gets farther, the size of the projected image also gets larger. However, this limitation in spatial resolution can be overcome with our new projector that supports much higher resolution (up to 4k), compared to the old one (maximum 1024 × 786 pixels), increasing the projected resolution more than threefold (8–15 pixels/degree).

Regardless of these limitations, our full-field scanning method provides promising new research avenues that can be explored in future investigations. One such avenue is to explore how the brain represents the spatial scale of a view in a more ecologically valid manner. Traditionally, we study object-focused views by cropping a picture closely to an object, eliminating all contextual peripheral visual information. However, this picture editing approach does not reflect how we actually experience the world, as we continuously receive visual information from the periphery even when focusing on an object. By simply moving the camera position (as an agent moves in an environment) and maintaining the same wide field-of-view, the spatial scale of the view is naturally determined by the distance between the focused object and the camera (agent). This positions us to investigate how we obtain a sense of object-focused view in the real visual world. Moreover, this method allows us to re-examine previous studies on various aspects of spatial representation in the brain. We can revisit how the continuous dimension of space is represented from an object-focused view to a far-scale navigable scene view 61 , how intermediate-scale scenes (e.g., a view of a chopping board) are represented in the brain 62 , and how the memory of a view is biased depending on the depicted spatial scale 63 , 64 . Importantly, this can be done while isolating field-of-view manipulation (e.g., cropping) from viewing distance manipulation.

Another promising research direction is to investigate the role of peripheral vision in computing one’s body position (relative to objects or environments) in complex, dynamically moving situations. This task is crucial for activities ranging from maneuvering a vehicle and playing sports to everyday walking and navigation. For this, extracting relevant visual cues such as optic flow, and sensitivity to the peripheral visual field in particular would be important. Notably, brain regions involved in these processes, such as the peripheral POS, human V6, prostriata, and potentially OPA 17 , 22 , 29 , are spatially adjacent along the POS. Full-field scanning offers a unique opportunity to directly stimulate these regions. This approach can enhance our understanding of how these areas interact and contribute to ego-motion computation, with wide-reaching implications for applied vision research.

The present findings reveal that classic scene regions are modulated by structured image and scene content, over dramatic changes in visual size, suggesting that they are tuned to particular higher-order image statistics rather than to any peripheral stimulation. Broadly, this study demonstrates how full-field neuroimaging allows us to investigate visual perception under more realistic, immersive experiences.

Participants

Twenty-two participants were recruited from the Harvard University Public Study Pool (10 females aged 20–54 years). All participants completed Experiment 1 (retinotopy protocol), ten participants in Experiment 2, and twelve participants in Experiment 3. All participants had normal or corrected-to-normal vision, gave informed consent, and were financially compensated. The experiments were performed in accordance with relevant guidelines and regulations and all procedures were approved by the Harvard University Human Subjects Institutional Review Board.

To enable ultra-wide-angle projection during scanning, several modifications were made to the typical scanning setup. In order to achieve an unobstructed view for the participant, we did not attach the top head coil and scanned only with the bottom head coil. Instead, we placed a custom-built curved screen right above the participant’s head. The screen was built with 3D-printed plastic parts and acrylic solvent. The curved shape was maintained by gluing a polystyrene sheet (1/16 inch thick) to a custom-fabricated acrylic hull (Supplementary Fig.  3 b). The radius of the cylindrical screen was 11 inches. The screen was made as large as possible while remaining rigidly supported and still fitting inside the MRI bore (about 12-inch radius). The one-inch difference allowed for the supporting ribs of the hull and a bit of clearance when moving in and out of the bore. Adjustable “legs” were attached at the bottom of the screen with nylon screws, and these legs were slotted into the scanner bed, allowing the screen to be securely anchored. Design files of the screen can be downloaded at https://jpark203.github.io/fullfield-neuroimaging/screen .

We also removed the standard flat projection screen at the back of the scanner bore. We bounced the projected image off of a pair of angled mirrors installed near the projector, directly into this curved screen inside the bore (Supplementary Fig.  3 a, Fig.  1) . For this, we constructed an inverted periscope. A pair of front surface mirrors were supported on a non-ferromagnetic stand. The lower mirror remains fixed, and the upper mirror is hinged. Tilting the upper mirror up removes the periscope from the projection path. With the periscope in place, the projector appears to originate from a virtual point further back and below the floor of the room.

Since this step changed how far the image on the screen is cast from the projector, we also adjusted the focus setting of the projector. Next, we used a reference image that was warped to fit the screen to check whether the image was accurately projected on the curved screen. If necessary, we carefully adjusted the projector position and/or the mirror angle. After this initial calibration stage, we refined the screen setup after a participant was put inside the scanner. First, we asked the participant to adjust their head position such that they were looking directly toward the center fixation mark on the screen. Second, we further adjusted the focus setup of the projector based on individual participants’ feedback. Overall, we allocated ~15–20 minutes of additional time for setting up the full-field scanning.

Image projection

To increase the spatial extent of stimulus, our goal was to project an image onto the inner wall of the cylinder bore. Ideally, the projector would be incident on the screen at 90 physical degrees. The physical geometry of the scanner bore makes this nearly impossible. The geometry of the room and our projection path are schematized in Supplementary Fig.  3 . The next best solution would be to place the projector (or the final mirror) closer to the cylinder bore in order to obtain the steepest angle possible. We did not pursue this route because any alterations must be minimally intrusive to alter any other ongoing study, as the MRI serves many labs.

If we projected directly onto the MRI bore, the light rays would be incident at just over 18 physical degrees. This shallow angle results in large distortion along the vertical (Y) axis of the projected image. To somewhat mitigate this, we angled the projection screen. Rather than being parallel to the magnet bore, we tilted it by 10 physical degrees. The edge of the screen at the top of the subject’s head nearly touches the bore. The screen edge near their collarbone is closer to the subject than the bore. Tilting angles larger than 10 physical degrees were ruled out for reasons of comfort–eye strain, feelings of confinement, etc. Effectively, this leads to the projector being angled slightly over 28 physical degrees relative to the screen (i.e., combining the tilted angle of the mirror and the screen).

As a result, approximately 1/3 of the Y pixels of the projector fall onto the screen, limiting our vertical resolution to 284 pixels rather than the native 768. In the case of the x pixels, about 828 pixels fall onto the screen, out of the native 1024 pixels (Fig.  2 , Supplementary Fig.  3 a). Pixels that did not intercept the display screen were set to black.

The visual angle of the display screen ranges from 168–182 degrees in width and 106–117 degrees in height (Supplementary Fig.  3 c). This variation depends on the distance between the participant’s eyes and the screen, which is affected by head size and head cushion options, for distances between 13.5–16.5 cm. For the stimulus size to be reported in the manuscript, we picked the middle viewing distance (15 cm) and calculated a stimulus angular extent. Perspective estimates did not take into account subject variability or binocularity.

The resolution of our current screen was 4.6–4.9 pixels per degree in width and 2.4–2.7 pixels/degree in height. It is noteworthy that the current limits on the low resolution can be overcome by our new projector, which has a much higher maximum resolution (4k). For example, if we keep the same aspect ratio of 4:3 (3200 × 2400), the pixels/degree will increase by the scaling factor of 3.125 (i.e., 2400/768 = 3.125).

Computational image warping

Because of the curvature and angle of the screen, all projected images were first computationally warped using a custom function to compensate for the geometry of the curved screen. Specifically, we developed a computational method that transforms a regular, rectangular image (1024 × 768 pixels; 4:3 aspect ratio) into a curved shape that matches the size and curvature of our custom-built screen. The transformed image on the cylindrical display surface preserved the same original aspect ratio (4:3) as it is measured 58.5 cm (arc length) × 44 cm (linear). Our image-warping algorithm allowed us to project the images onto the cylindrical screen without stretch or distortion; similar to the real-world action of pasting a sheet of wallpaper onto a cylindrical wall.

To link the warping algorithm parameters to the physical set up, we developed a calibration procedure, in which we use an MR-compatible mouse to obtain the × and y coordinates of the projector image that correspond with the three points along the screen outline (e.g., measuring points along both the top and bottom of screen curvature separately, as the bottom screen was slightly narrower than the top). This resulted in a 2d mapping, which takes an original image, and then resizes and warps it to be positioned directly into the part of the projected image that is being projected onto the screen (Fig.  2) .

Signal quality check

Several quality assurance tests were conducted with and without the top head coil separately, to check how much fMRI signal was impacted by removing the top head coil. First, we ran the CoilQA sequence that calculates and provides an Image SNR map. Second, we ran one of our BOLD protocols (i.e., one of the functional runs), computed tSNR maps, and examined BOLD quality check results. Third, we also ran the T1-weighted scan for a qualitative comparison between the two cases. The test results are reported in Supplementary Fig.  1 .

Additionally, we also computed tSNR within each ROI. For this, we preprocessed the data using the identical protocol as the main experiments and normalized it into Talairach space. The voxel-wise tSNR was calculated by dividing the mean by the standard deviation of time-course data. Then, we extracted voxels for each ROI, and averaged their tSNRs to get an ROI tSNR value. The comparison between with and without the top head coil is reported in Supplementary Fig.  2 .

Rendering full-field views from virtual 3D environments

Computer-generated (CGI) environments were generated using the Unity video game engine (Unity Technologies, Version 2017.3.0). We constructed twenty indoor environments, reflecting a variety of semantic categories (e.g., kitchens, bedrooms, laboratories, cafeterias, etc.). All rooms had the same physical dimensions (4 width × 3 height × 6 depth arbitrary units in Unity), with an extended horizontal surface along the back wall, containing a centrally positioned object. Each environment was additionally populated with the kinds of objects typically encountered in those locations, creating naturalistic CGI environments. These environments were also used in refs. 61 , 63 .

Next, for each environment, we rendered an image view, set to mimic the view of an adult standing in a room looking at the object on the back counter/surface. During the development of these protocols, we found that it was important to get the camera parameters related to the field of view (FOV) right to feel as if you were standing in the room with objects having their familiar sizes; otherwise, viewers were prone to experience distortions of space. Here the camera FOV was fixed at 105 degrees in height and 120.2 degrees in width. This FOV was chosen based on the chord angle of our physical screen (120 deg) and empirical testing by experimenters. Since there was no ground truth for the size of virtual reality environments (e.g., how large the space should be), experimenters compared a few different FOVs and made subjective judgments on which parameter feels most natural. Relatedly, we set the camera height to be 1.6 (arbitrary units), and tilted the camera angle down (mean rotation angle = 5.2 deg, s.d. = 0.5 deg, across 20 environments), so that the center object was always at the center of the image. For these stimuli, we positioned the camera at the back of the environment, to give a view of the entire room. Each image was rendered at 1024 × 768 pixels.

Experiment 1

In the retinotopy runs (5.8 min, 174 TRs), there were 7 conditions: horizontal bands, vertical bands, and five levels of eccentricities (e.g., from foveal stimulation to far-peripheral stimulation). A center circle was 1.8 degrees radius, and the inner and outer rings of the rest of the conditions were 2.0–5.6 degrees, 6.3–16.5 degrees, 18.5–50.3 degrees, and >55.3 degrees radius. All stimuli cycled between states of black-and-white, white-and-black, and randomly colored, at 4Hz. Each run consisted of 7 blocks per condition (6-sec block), with seven 6-sec fixation blocks interleaved throughout the experiment. An additional 6-sec fixation block was added at the beginning and the end of the run. Participants were asked to maintain fixation and press a button when the fixation dot turned blue, which happened at a random time once per block.

Experiment 2

In Experiment 2, participants completed 8 runs of the main protocol (one participant completed 6, and two participants completed 5 runs) and 3 retinotopy runs (two participants completed 2 runs).

In the main protocol, there were 7 stimulus conditions. (1) Full-field scenes: 15 full-field scene images were chosen (randomly selected from the 20 total environments). (2) Full-field Phase-scrambled image. First, the images were fast Fourier transformed (FFT) to decompose them into amplitude and phase spectrum. Then, the phase spectrum was randomized by adding random values to the original phase spectrum. The resulting phase spectrum was combined with the amplitude spectrum, then transformed back to an image using an inverse Fourier transform 65 . (3) Postcard scenes. These images were generated by rescaling the full-field scenes. Instead of cropping the central portion of the original image, an entire image was rescaled from 1024 × 786 pixels to 205 × 154 pixels (44 degrees wide). This rescaled image was positioned at the center, and the rest of area around it was filled with the background color, such that the size of whole image (i.e., small scene at the center with the padding around it) was kept the same as the original image (1024 × 768 pixels). (4) Postcard-scrambled scenes. The same rescaling procedure was followed for the phase-scrambled scenes. The final three conditions consisted of fifteen images from each category of (5) faces, (6) big animate objects, and (7) small inanimate objects. They were rescaled to fit a bounding box (171 × 129 pixels; 37 degrees wide) with white background color. This bounding box was positioned at the center with the padding, so that the size of an output image is 1024 × 768 pixels.

A single run of the main protocol was 6.5 min in duration (195 TRs) and was a classic on-off blocked design. A condition block was 12 sec, and was always followed by 6 sec fixation period. Within each block, six trials from one condition were presented. Each trial consisted of 1.5 sec stimulus presentation and 500 ms blank screen. The stimulus duration was chosen to be a little longer than the typical scanning, because flashing full-field images too fast can be uncomfortable and may cause nausea. Among those six images, five were unique images, and one of those images was randomly chosen and repeated twice in a row. Participants were instructed to press a button when they saw the repeated image (one-back repetition detection task). The presentation order of blocks was pseudo-randomized for each run as follows. Seven conditions within an epoch were randomized 3 times independently and concatenated with a constraint that the same condition cannot appear in two successive blocks. Thus, each of 7 condition blocks were presented 3 times per run. Fifteen unique images per condition were randomly split across those three blocks, for each run.

Experiment 3

In Experiment 3, participants completed 8 runs of the main protocol (one participant completed 7, and five participants completed 6 runs), 2 runs of classic category localizer (two participants completed 1 run, and two participants did not complete any localizers and were excluded from ROI analyses), and 2 retinotopy runs (two participants completed 3 runs).

In the main protocol of Experiment 3, stimuli were varied with 2 factors: image content (scenes, phase-scrambled scenes, face arrays, object arrays), and scotoma size (0, 29, 58, 88, 140 degrees in diameter). The scene images were captured from 20 virtual environments built in Unity, using the same camera parameters as in Experiment 2. For face and object conditions, 58 individual faces and objects were collected. We matched the luminance across scenes, faces, and objects, by equating the luminance histograms using Color SHINE toolbox 66 . The phase-scrambled scenes were generated from the luminance-matched scenes, using the same parameters as in Experiment 1.

Face and object arrays were generated with those luminance-matched images. For each face array, 13 faces were randomly drawn from a pool of 58 faces (half male). These faces were arranged along 3 levels of eccentricity circles. The size of individual faces and the number of faces was adjusted for each eccentricity, to account for cortical magnification and to avoid crowding effect. At the smallest eccentricity, 3 faces were rescaled to the size of 113-pixel diameter; at the middle eccentricity, 6 faces were rescaled to the size of 178-pixel diameter; at the largest eccentricity, 4 faces were rescaled to the size of 295-pixel diameter. The largest faces were positioned at 4 corners of the image, and the rest of faces were equally distanced along the eccentricity circle, with random jitters applied to individual face locations. Object arrays were generated using the same procedure. This step resulted in 20 face arrays and 20 object arrays. After making those base stimuli with 4 different image content (scenes, phase-scrambled scenes, face arrays, object arrays), we generated scotoma conditions by applying scotoma masks with 5 levels: 0 (i.e., no mask), 29, 58, 88, and 140 degrees in diameter. In total, 400 unique stimuli were generated across 20 conditions.

The main protocol was 6.9 min in duration (208 TRs), and used a block design, with 20 conditions presented twice per run. In each condition block (8 sec), five trials from one condition were presented. Each trial consisted of 1.1 sec stimulus presentation, followed by 500 ms blank screen. A fixation (black and white bullseye) was presented at the center of screen throughout an entire block. Among those five images in a block, four were unique images, and one of those images was randomly chosen and repeated twice in a row. Participants were asked to press a button when they detected the repetition. The presentation order of blocks in each run was randomized within each epoch. One epoch consisted of one block from each of 20 conditions and 5 resting blocks (8 sec). For each epoch, 20 unique images per condition were randomly split across 5 scotoma conditions. This procedure was repeated twice and concatenated with a constraint that the same condition cannot appear in two successive blocks. Thus, each of 20 condition blocks were repeated twice per run.

The classic category localizer was 6.9 min (208 TRs) and consisted of four conditions: scenes, faces, objects, and scrambled objects. Ten blocks per condition were acquired within a run. In each condition block (8 sec), four unique images were selected, and one of those images was randomly chosen and repeated twice in a row. Participants performed the one-back repetition task. Each image was presented for 1.1 sec and followed by 500 ms blank. In each run, the block order was randomized within each epoch, which consisted of one block from each condition and one fixation block (8 sec). This procedure was repeated ten times, and the block orders were concatenated across the epochs.

Additionally, the same retinotopy protocol from Experiment 2 was run. All stimuli presentation and the experiment program were produced and controlled by MATLAB R2020b and Psychophysics Toolbox (3.0.17) 67 , 68 .

Behavioral recognition task

To test whether participants can recognize a basic category of stimuli, a 2-alternative-forced choice (2AFC) was performed inside the scanner during an MPRAGE protocol. Only the face arrays and object arrays with scotomas were tested. Each array was presented for 1.1 sec, which was the same duration used in the main protocol. Then, participants were asked to indicate whether the stimulus was faces or objects, using a response button box.

fMRI data acquisition

All neuroimaging data were collected at the Harvard Center for Brain Sciences using the bottom half (20 channels) of a 32-channel phased-array head coil with a 3T Siemens Prisma fMRI Scanner. High-resolution T1-weighted anatomical scans were acquired using a 3D multi-echo MPRAGE protocol 69 (176 sagittal slices; FOV = 256 mm; 1 × 1 × 1 mm voxel resolution; gap thickness = 0 mm; TR = 2530 ms; TE = 1.69, 3.55, 5.41, and 7.27 ms; flip angle = 7°). Blood oxygenation level-dependent (BOLD) contrast functional scans were obtained using a gradient echo-planar T2* sequence (87 oblique axial slices acquired at a 25° angle off of the anterior commissure-posterior commissure line; FOV = 211 mm; 1.7 × 1.7 × 1.7 mm voxel resolution; gap thickness = 0 mm; TR = 2000 ms; TE = 30 ms, flip angle = 80°, multiband acceleration factor = 3, in-plane acceleration factor = 2) 70 , 71 , 72 , 73 .

fMRI data analysis and preprocessing

The fMRI data were analyzed with BrainVoyager 21.2.0 software (Brain Innovation) with custom Matlab scripting. Preprocessing included slice-time correction, linear trend removal, 3D motion correction, temporal high-pass filtering, and spatial smoothing (4mm FWHM kernel). The data were first aligned to the AC-PC axis, then transformed into the standardized Talairach space (TAL). Three-dimensional models of each participant’s cortical surface were generated from the high-resolution T1-weighted anatomical scan using the default segmentation procedures in FreeSurfer. For visualizing activations on inflated brains, the segmented surfaces were imported back into BrainVoyager and inflated using the BrainVoyager surface module. Gray matter masks were defined in the volume based on the Freesurfer cortex segmentation.

A general linear model (GLM) was fit for each participant using BrainVoyager. The design matrix included regressors for each condition block and 6 motion parameters as nuisance regressors. The condition regressors were constructed based on boxcar functions for each condition, convolved with a canonical hemodynamic response function (HRF), and were used to fit voxel-wise time course data with percent signal change normalization and correction for serial correlations. The beta weights from the GLM were used as measures of activation to each condition for all subsequent analyses.

Regions of interest (ROIs)

Experiment 2 did not have separate localizer runs. So, we split the main runs into two sets and used the half of runs to localize ROIs and the other half to extract data for subsequent analyses. We defined ROIs separately in each hemisphere in each participant, using condition contrasts implemented in subject-specific GLMs. Three scene-selective areas were defined using [Postcard Scenes–Faces] contrast ( p  < 0.0001). Specifically, the PPA was defined by locating the cluster between posterior parahippocampal gyrus and lingual gyrus, the RSC was defined by locating the cluster near the posterior cingulate cortex, and the OPA was defined by locating the cluster near transverse occipital sulcus. The FFA was defined using [Faces–Postcard Scene] contrast ( p  < 0.0001). The early visual areas (EVA; V1–V3) were defined manually on inflated brain, based on the contrast of [Horizontal–Vertical] meridians from the retinotopy runs.

In Experiment 3, independent localizer runs were used to define ROIs. We defined the PPA, RSC, and OPA using [Scenes–Faces] contrast ( p  < 0.0001). The FFA was defined using [Faces–Scenes] contrast ( p  < 0.001). The lateral occipital complex (LOC) was defined using [Objects–Scrambled Objects] contrast ( p  < 0.0001). Finally, the early visual areas (EVA; V1–V3) were defined manually on the inflated brain based on the contrast of [Horizontal–Vertical] meridians from the retinotopy runs. All ROIs were defined separately in each hemisphere of each participant.

Eccentricity preference map

To examine a topographic mapping of the eccentricity map, we calculated a group-level preference map. First, responses to each of 5 levels of eccentricities were extracted in each voxel from single-subject GLMs and then averaged over subjects. For each voxel, a condition showing the highest group-average response was identified as the preferred condition. The degree of preference was computed by taking the response differences between the most preferred condition and the next most preferred condition. For visualization, we colored each voxel with a color hue corresponding to the preferred condition, with a color intensity reflecting the degree of preference. The resulting preference map was projected onto the cortical surface of a sample participant. The same preference mapping procedures were used to generate individual subject preference mapping as well.

Reporting summary

Further information on research design is available in the  Nature Portfolio Reporting Summary linked to this article.

Data availability

Preprocessed fMRI data, design matrices, ROI masks, and all stimuli before the image transformation are available in the Open Science Framework repository ( https://osf.io/5hsbv ).  Source data are provided with this paper.

Code availability

Image transformation scripts are available on Github ( https://github.com/jpark203/FullField-ImageWarping ; https://doi.org/10.5281/zenodo.11113136 ) and on the method website ( https://jpark203.github.io/fullfield-neuroimaging ).

Strasburger, H. Seven myths on crowding and peripheral vision. i-Percept. 11 , 2041669520913052 (2020).

Google Scholar  

Epstein, R. A. & Baker, C. I. Scene perception in the human brain. Annu. Rev. Vis. Sci. 5 , 373–397 (2019).

Article   PubMed   PubMed Central   Google Scholar  

Dilks, D. D., Kamps, F. S. & Persichetti, A. S. Three cortical scene systems and their development. Trends. Cogn. Sci. 26 , 117–127 (2022).

Epstein, R. & Kanwisher, N. A cortical representation of the local visual environment. Nature 392 , 598–601 (1998).

Article   ADS   CAS   PubMed   Google Scholar  

O’Craven, K. M. & Kanwisher, N. Mental imagery of faces and places activates corresponding stimulus-specific brain regions. J. Cogn. Neurosci. 12 , 1013–1023 (2000).

Article   PubMed   Google Scholar  

Silson, E. H., Steel, A. D. & Baker, C. I. Scene-selectivity and retinotopy in medial parietal cortex. Front. Hum. Neurosci. 10 , 412 (2016).

Grill-Spector, K. The neural basis of object perception. Curr. Opin. Neurobiol. 13 , 159–166 (2003).

Article   CAS   PubMed   Google Scholar  

Dilks, D. D., Julian, J. B., Paunov, A. M. & Kanwisher, N. The occipital place area is causally and selectively involved in scene perception. J. Neurosci. 33 , 1331–1336 (2013).

Walther, D. B., Caddigan, E., Fei-Fei, L. & Beck, D. M. Natural scene categories revealed in distributed patterns of activity in the human brain. J. Neurosci. 29 , 10573–10581 (2009).

Article   CAS   PubMed   PubMed Central   Google Scholar  

Epstein, R. A. & Morgan, L. K. Neural responses to visual scenes reveals inconsistencies between fmri adaptation and multivoxel pattern analysis. Neuropsychologia 50 , 530–543 (2012).

Kornblith, S., Cheng, X., Ohayon, S. & Tsao, D. Y. A network for scene processing in the macaque temporal lobe. Neuron 79 , 766–781 (2013).

Park, J. & Park, S. Conjoint representation of texture ensemble and location in the parahippocampal place area. J. Neurophysiol. 117 , 1595–1607 (2017).

Oliva, A. & Torralba, A. Modeling the shape of the scene: a holistic representation of the spatial envelope. Int. J. Comput. Vis. 42 , 145–175 (2001).

Article   Google Scholar  

Bonner, M. F. & Epstein, R. A. Coding of navigational affordances in the human visual system. Proc. Natl. Acad. Sci. 114 , 4793–4798 (2017).

Article   ADS   CAS   PubMed   PubMed Central   Google Scholar  

Marchette, S. A., Vass, L. K., Ryan, J. & Epstein, R. A. Anchoring the neural compass: coding of local spatial reference frames in human medial parietal lobe. Nat. Neurosci. 17 , 1598–1606 (2014).

Persichetti, A. S. & Dilks, D. D. Dissociable neural systems for recognizing places and navigating through them. J. Neurosci. 38 , 10295–10304 (2018).

Kamps, F. S., Lall, V. & Dilks, D. D. The occipital place area represents first-person perspective motion information through scenes. Cortex 83 , 17–26 (2016).

Robertson, C. E., Hermann, K. L., Mynick, A., Kravitz, D. J. & Kanwisher, N. Neural representations integrate the current field of view with the remembered 360 panorama in scene-selective cortex. Curr. Biol. 26 , 2463–2468 (2016).

Ferrara, K. & Park, S. Neural representation of scene boundaries. Neuropsychologia 89 , 180–190 (2016).

Park, J. & Park, S. Coding of navigational distance and functional constraint of boundaries in the human scene-selective cortex. J. Neurosci. 40 , 3621–3630 (2020).

Ellis, C. et al. Re-imagining fmri for awake behaving infants. Nat. Commun. 11 , 4523 (2020).

Pitzalis, S. et al. Wide-field retinotopy defines human cortical visual area v6. J. Neurosci. 26 , 7962–7973 (2006).

Stenbacka, L. & Vanni, S. fmri of peripheral visual field representation. Clin. Neurophysiol. 118 , 1303–1314 (2007).

Yan, T., Jin, F., He, J. & Wu, J. Development of a wide-view visual presentation system for visual retinotopic mapping during functional mri. J. Magn. Reson. Imaging 33 , 441–447 (2011).

Wu, J., Yan, T., Zhang, Z., Jin, F. & Guo, Q. Retinotopic mapping of the peripheral visual field to human visual cortex by functional magnetic resonance imaging. Hum. Brain Mapp. 33 , 1727–1740 (2012).

Arnoldussen, D. M., Goossens, J. & van den Berg, A. V. Adjacent visual representations of self-motion in different reference frames. Proc. Natl Acad. Sci. 108 , 11668–11673 (2011).

Wu, J. et al. Development of a method to present wide-view visual stimuli in mri for peripheral visual studies. J. Neurosci. Methods 214 , 126–136 (2013).

Greco, V. et al. A low-cost and versatile system for projecting wide-field visual stimuli within fmri scanners. Behav. Res. Methods 48 , 614–620 (2016).

Mikellidou, K. et al. Area prostriata in the human brain. Curr. Biol. 27 , 3056–3060 (2017).

Levy, I., Hasson, U., Avidan, G., Hendler, T. & Malach, R. Center–periphery organization of human object areas. Nat. Neurosci. 4 , 533–539 (2001).

Silson, E. H., Chan, A. W.-Y., Reynolds, R. C., Kravitz, D. J. & Baker, C. I. A retinotopic basis for the division of high-level scene processing between lateral and ventral human occipitotemporal cortex. J. Neurosci. 35 , 11921–11935 (2015).

Baldassano, C., Esteva, A., Fei-Fei, L. & Beck, D. M. Two distinct scene-processing networks connecting vision and memory. Eneuro 3 , 5 (2016).

McCotter, M., Gosselin, F., Sowden, P. & Schyns, P. The use of visual information in natural scenes. Vis. Cogn. 12 , 938–953 (2005).

Park, S., Brady, T. F., Greene, M. R. & Oliva, A. Disentangling scene content from spatial boundary: complementary roles for the parahippocampal place area and lateral occipital complex in representing real-world scenes. J. Neurosci. 31 , 1333–1340 (2011).

Loschky, L. et al. The contributions of central and peripheral vision to scene gist recognition with a 180° visual field. J. Vis. 15 , 570–570 (2015).

Hasson, U., Levy, I., Behrmann, M., Hendler, T. & Malach, R. Eccentricity bias as an organizing principle for human high-order object areas. Neuron 34 , 479–490 (2002).

Harel, A., Kravitz, D. J. & Baker, C. I. Deconstructing visual scenes in cortex: gradients of object and spatial layout information. Cereb. Cortex 23 , 947–957 (2013).

Choo, H. & Walther, D. B. Contour junctions underlie neural representations of scene categories in high-level human visual cortex. Neuroimage 135 , 32–44 (2016).

Cant, J. S. & Xu, Y. Object ensemble processing in human anterior-medial ventral visual cortex. J. Neurosci. 32 , 7685–7700 (2012).

Stansbury, D. E., Naselaris, T. & Gallant, J. L. Natural scene statistics account for the representation of scene categories in human visual cortex. Neuron 79 , 1025–1034 (2013).

Van Essen, D. & Drury, H. Structural and functional analyses of human cerebral cortex using a surface-based atlas. J. Neurosci. 17 , 7079–7102 (1997).

Hasson, U., Harel, M., Levy, I. & Malach, R. Large-scale mirror-symmetry organization of human occipito-temporal object areas. Neuron 37 , 1027–1041 (2003).

Steel, A., Billings, M. M., Silson, E. H. & Robertson, C. E. A network linking scene perception and spatial memory systems in posterior cerebral cortex. Nat. Commun. 12 , 2632 (2021).

Pitzalis, S. et al. Human v6: the medial motion area. Cereb. Cortex 20 , 411–424 (2010).

Cardin, V. & Smith, A. T. Sensitivity of human visual and vestibular cortical regions to egomotion-compatible visual stimulation. Cereb. Cortex 20 , 1964–1973 (2010).

Malach, R., Levy, I. & Hasson, U. The topography of high-order human object areas. Trends Cogn. Sci. 6 , 176–184 (2002).

Baldassano, C., Fei-Fei, L. & Beck, D. M. Pinpointing the peripheral bias in neural scene-processing networks during natural viewing. J. Vis. 16 , 9–9 (2016).

Kamps, F. S., Hendrix, C. L., Brennan, P. A. & Dilks, D. D. Connectivity at the origins of domain specificity in the cortical face and place networks. Proc. Natl Acad. Sci. 117 , 6163–6169 (2020).

Arcaro, M. J. & Livingstone, M. S. A hierarchical, retinotopic proto-organization of the primate visual system at birth. Elife 6 , e26196 (2017).

Finzi, D. Differential spatial computations in ventral and lateral face-selective regions are scaffolded by structural connections. Nat. Commun. 12 , 2278 (2021).

Kay, K. N., Weiner, K. S. & Grill-Spector, K. Attention reduces spatial uncertainty in human ventral temporal cortex. Curr. Biol. 25 , 595–600 (2015).

Rima, S. & Schmid, M. C. V1-bypassing thalamo-cortical visual circuits in blindsight and developmental dyslexia. Curr. Opin. Physiol. 16 , 14–20 (2020).

Beltramo, R. & Scanziani, M. A collicular visual cortex: neocortical space for an ancient midbrain visual structure. Science 363 , 64–69 (2019).

Pitcher, D., Dilks, D. D., Saxe, R. R., Triantafyllou, C. & Kanwisher, N. Differential selectivity for dynamic versus static information in face-selective cortical regions. Neuroimage 56 , 2356–2363 (2011).

Pitcher, D. & Ungerleider, L. G. Evidence for a third visual pathway specialized for social perception. Trends Cogn. Sci. 25 , 100–110 (2021).

Frey, M., Nau, M. & Doeller, C. F. Magnetic resonance-based eye tracking using deep neural networks. Nat. Neurosci. 24 , 1772–1779 (2021).

Tootell, R. B. et al. Functional analysis of human mt and related visual cortical areas using magnetic resonance imaging. J. Neurosci. 15 , 3215–3230 (1995).

Cheng, K., Fujita, H., Kanno, I., Miura, S. & Tanaka, K. Human cortical regions activated by wide-field visual motion: an h2 (15) o pet study. J. Neurophysiol. 74 , 413–427 (1995).

Stenbacka, L. & Vanni, S. Central luminance flicker can activate peripheral retinotopic representation. Neuroimage 34 , 342–348 (2007).

Nasr, S. et al. In vivo functional localization of the temporal monocular crescent representation in human primary visual cortex. Neuroimage 209 , 116516 (2020).

Park, J., Josephs, E. & Konkle, T. Ramp-shaped neural tuning supports graded population-level representation of the object-to-scene continuum. Sci Rep. 12 , 18081 (2022).

Josephs, E. L. & Konkle, T. Large-scale dissociations between views of objects, scenes, and reachable-scale environments in visual cortex. Proc. Natl Acad. Sci. 117 , 29354–29362 (2020).

Park, J., Josephs, E. & Konkle, T. Systematic transition from boundary extension to contraction along an object-to-scene continuum. J. Vis. 24 , 9–9 (2024).

Bainbridge, W. A. & Baker, C. I. Boundaries extend and contract in scene memory depending on image properties. Curr. Biol. 30 , 537–543 (2020).

Ragni, F., Tucciarelli, R., Andersson, P. & Lingnau, A. Decoding stimulus identity in occipital, parietal and inferotemporal cortices during visual mental imagery. Cortex 127 , 371–387 (2020).

Dal Ben, R. Shine_color: controlling low-level properties of colorful images. MethodsX . 11 , 102377 (2021).

Brainard, D. H. The psychophysics toolbox. Spat. Vis. 10 , 433–436 (1997).

Pelli, D. G. & Vision, S. The videotoolbox software for visual psychophysics: transforming numbers into movies. Spat. Vis. 10 , 437–442 (1997).

Van der Kouwe, A. J., Benner, T., Salat, D. H. & Fischl, B. Brain morphometry with multiecho mprage. Neuroimage 40 , 559–569 (2008).

Moeller, S. et al. Multiband multislice ge-epi at 7 tesla, with 16-fold acceleration using partial parallel imaging with application to high spatial and temporal whole-brain fmri. Magn. Reson. Med. 63 , 1144–1153 (2010).

Feinberg, D. A. et al. Multiplexed echo planar imaging for sub-second whole brain fmri and fast diffusion imaging. PloS One 5 , e15710 (2010).

Setsompop, K. & Gagoski, B. A. et al. Blipped-controlled aliasing in parallel imaging for simultaneous multislice echo planar imaging with reduced g-factor penalty. Magn. Reson. Med. 67 , 1210–1224 (2012).

Xu, J. et al. Evaluation of slice accelerations using multiband echo planar imaging at 3 t. Neuroimage 83 , 991–1001 (2013).

Download references

Acknowledgements

Research reported in this study was supported by the Harvard Brain Science Initiative Postdoc Pioneers Grant awarded to J.P. and the National Eye Institute of the National Institutes of Health under Award Number R21EY031867 awarded to T.K. The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health. This research was carried out at the Harvard Center for Brain Science and involved the use of instrumentation supported by the NIH Shared Instrumentation Grant Program (S10OD020039). We acknowledge the University of Minnesota Center for Magnetic Resonance Research for the use of the multiband-EPI pulse sequences. We also thank MCB Graphics at Harvard University for their assistance with the graphics in figures.

Author information

Authors and affiliations.

Department of Psychology, Harvard University, Cambridge, MA, USA

Jeongho Park & Talia Konkle

Center for Brain Science, Harvard University, Cambridge, MA, USA

Edward Soucy, Jennifer Segawa, Ross Mair & Talia Konkle

Department of Radiology, Harvard Medical School, Boston, MA, USA

Department of Radiology, Massachusetts General Hospital, Boston, MA, USA

Kempner Institute for Biological and Artificial Intelligence, Harvard University, Boston, MA, USA

Talia Konkle

You can also search for this author in PubMed   Google Scholar

Contributions

J.P. and T.K. designed the research, interpreted data, and wrote the paper. E.S. designed and constructed the physical apparatus for image projection and developed the computational warping algorithm. J.P. and J.S. collected data. R.M. performed an MRI signal quality assessment. All data preprocessing and experimental analyses were performed by J.P.

Corresponding author

Correspondence to Jeongho Park .

Ethics declarations

Competing interests.

The authors declare no competing interests.

Peer review

Peer review information.

Nature Communications thanks Jody Culham, Mark Lescroart and the other, anonymous, reviewer(s) for their contribution to the peer review of this work. A peer review file is available.

Additional information

Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary information

Supplementary information, peer review file, reporting summary, source data, source data, rights and permissions.

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ .

Reprints and permissions

About this article

Cite this article.

Park, J., Soucy, E., Segawa, J. et al. Immersive scene representation in human visual cortex with ultra-wide-angle neuroimaging. Nat Commun 15 , 5477 (2024). https://doi.org/10.1038/s41467-024-49669-0

Download citation

Received : 01 September 2023

Accepted : 13 June 2024

Published : 28 June 2024

DOI : https://doi.org/10.1038/s41467-024-49669-0

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

By submitting a comment you agree to abide by our Terms and Community Guidelines . If you find something abusive or that does not comply with our terms or guidelines please flag it as inappropriate.

Quick links

• Explore articles by subject
• Guide to authors
• Editorial policies

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

• X (Twitter)

Painting Pictures with Data: The Power of Visual Representations

Picture this. A chaotic world of abstract concepts and complex data, like a thousand-piece jigsaw puzzle. Each piece, a different variable, a unique detail.

Alone, they’re baffling, nearly indecipherable.

But together? They’re a masterpiece of visual information, a detailed illustration.

American data pioneer Edward Tufte , a notable figure in the graphics press, believed that the art of seeing is not limited to the physical objects around us. He stated, “The commonality between science and art is in trying to see profoundly – to develop strategies of seeing and showing.”

It’s in this context that we delve into the world of data visualization. This is a process where you create visual representations that foster understanding and enhance decision making.

It’s the transformation of data into visual formats. The information could be anything from theoretical frameworks and research findings to word problems. Or anything in-between. And it has the power to change the way you learn, work, and more.

And with the help of modern technology, you can take advantage of data visualization easier than ever today.

What are Visual Representations?

Think of visuals, a smorgasbord of graphical representation, images, pictures, and drawings. Now blend these with ideas, abstract concepts, and data.

You get visual representations . A powerful, potent blend of communication and learning.

As a more formal definition, visual representation is the use of images to represent different types of data and ideas.

They’re more than simply a picture. Visual representations organize information visually , creating a deeper understanding and fostering conceptual understanding. These can be concrete objects or abstract symbols or forms, each telling a unique story. And they can be used to improve understanding everywhere, from a job site to an online article. University professors can even use them to improve their teaching.

But this only scratches the surface of what can be created via visual representation.

Types of Visual Representation for Improving Conceptual Understanding

Graphs, spider diagrams, cluster diagrams – the list is endless!

Each type of visual representation has its specific uses. A mind map template can help you create a detailed illustration of your thought process. It illustrates your ideas or data in an engaging way and reveals how they connect.

Here are a handful of different types of data visualization tools that you can begin using right now.

1. Spider Diagrams

Spider diagrams , or mind maps, are the master web-weavers of visual representation.

They originate from a central concept and extend outwards like a spider’s web. Different ideas or concepts branch out from the center area, providing a holistic view of the topic.

This form of representation is brilliant for showcasing relationships between concepts, fostering a deeper understanding of the subject at hand.

2. Cluster Diagrams

As champions of grouping and classifying information, cluster diagrams are your go-to tools for usability testing or decision making. They help you group similar ideas together, making it easier to digest and understand information.

They’re great for exploring product features, brainstorming solutions, or sorting out ideas.

3. Pie Charts

Pie charts are the quintessential representatives of quantitative information.

They are a type of visual diagrams that transform complex data and word problems into simple symbols. Each slice of the pie is a story, a visual display of the part-to-whole relationship.

Whether you’re presenting survey results, market share data, or budget allocation, a pie chart offers a straightforward, easily digestible visual representation.

4. Bar Charts

If you’re dealing with comparative data or need a visual for data analysis, bar charts or graphs come to the rescue.

Bar graphs represent different variables or categories against a quantity, making them perfect for representing quantitative information. The vertical or horizontal bars bring the data to life, translating numbers into visual elements that provide context and insights at a glance.

Visual Representations Benefits

1. deeper understanding via visual perception.

Visual representations aren’t just a feast for the eyes; they’re food for thought. They offer a quick way to dig down into more detail when examining an issue.

They mold abstract concepts into concrete objects, breathing life into the raw, quantitative information. As you glimpse into the world of data through these visualization techniques , your perception deepens.

You no longer just see the data; you comprehend it, you understand its story. Complex data sheds its mystifying cloak, revealing itself in a visual format that your mind grasps instantly. It’s like going from a two dimensional to a three dimensional picture of the world.

2. Enhanced Decision Making

Navigating through different variables and relationships can feel like walking through a labyrinth. But visualize these with a spider diagram or cluster diagram, and the path becomes clear. Visual representation is one of the most efficient decision making techniques .

Visual representations illuminate the links and connections, presenting a fuller picture. It’s like having a compass in your decision-making journey, guiding you toward the correct answer.

3. Professional Development

Whether you’re presenting research findings, sharing theoretical frameworks, or revealing historical examples, visual representations are your ace. They equip you with a new language, empowering you to convey your message compellingly.

From the conference room to the university lecture hall, they enhance your communication and teaching skills, propelling your professional development. Try to create a research mind map and compare it to a plain text document full of research documentation and see the difference.

4. Bridging the Gap in Data Analysis

What is data visualization if not the mediator between data analysis and understanding? It’s more than an actual process; it’s a bridge.

It takes you from the shores of raw, complex data to the lands of comprehension and insights. With visualization techniques, such as the use of simple symbols or detailed illustrations, you can navigate through this bridge effortlessly.

5. Enriching Learning Environments

Imagine a teaching setting where concepts are not just told but shown. Where students don’t just listen to word problems but see them represented in charts and graphs. This is what visual representations bring to learning environments.

They transform traditional methods into interactive learning experiences, enabling students to grasp complex ideas and understand relationships more clearly. The result? An enriched learning experience that fosters conceptual understanding.

6. Making Abstract Concepts Understandable

In a world brimming with abstract concepts, visual representations are our saving grace. They serve as translators, decoding these concepts into a language we can understand.

Let’s say you’re trying to grasp a theoretical framework. Reading about it might leave you puzzled. But see it laid out in a spider diagram or a concept map, and the fog lifts. With its different variables clearly represented, the concept becomes tangible.

Visual representations simplify the complex, convert the abstract into concrete, making the inscrutable suddenly crystal clear. It’s the power of transforming word problems into visual displays, a method that doesn’t just provide the correct answer. It also offers a deeper understanding.

How to Make a Cluster Diagram?

Ready to get creative? Let’s make a cluster diagram.

First, choose your central idea or problem. This goes in the center area of your diagram. Next, think about related topics or subtopics. Draw lines from the central idea to these topics. Each line represents a relationship.

While you can create a picture like this by drawing, there’s a better way.

Mindomo is a mind mapping tool that will enable you to create visuals that represent data quickly and easily. It provides a wide range of templates to kick-start your diagramming process. And since it’s an online site, you can access it from anywhere.

With a mind map template, creating a cluster diagram becomes an effortless process. This is especially the case since you can edit its style, colors, and more to your heart’s content. And when you’re done, sharing is as simple as clicking a button.

A Few Final Words About Information Visualization

To wrap it up, visual representations are not just about presenting data or information. They are about creating a shared understanding, facilitating learning, and promoting effective communication. Whether it’s about defining a complex process or representing an abstract concept, visual representations have it all covered. And with tools like Mindomo , creating these visuals is as easy as pie.

In the end, visual representation isn’t just about viewing data, it’s about seeing, understanding, and interacting with it. It’s about immersing yourself in the world of abstract concepts, transforming them into tangible visual elements. It’s about seeing relationships between ideas in full color. It’s a whole new language that opens doors to a world of possibilities.

The correct answer to ‘what is data visualization?’ is simple. It’s the future of learning, teaching, and decision-making.

Keep it smart, simple, and creative! The Mindomo Team

Related Posts

Top 5 Fishbone Diagram Templates You Need To Know About!

Mastering Your Mind: Exploring Effective Visualization Techniques

The Power of an Idea Map: Your Guide to Creative Thinking & Organizing Ideas

Innovation Unleashed: Mind Mapping vs Brainstorming in the Generation of Game-Changing Ideas

The Key to Success with Ingredients for a Fulfilling Life

Cracking the Code to Creative Thinking: Ignite Your Brain and Unleash Your Ideas

Write a comment cancel reply.

Save my name, email, and website in this browser for the next time I comment.

Suggestions or feedback?

MIT News | Massachusetts Institute of Technology

• Machine learning
• Social justice
• Black holes
• Classes and programs

Departments

• Aeronautics and Astronautics
• Brain and Cognitive Sciences
• Architecture
• Political Science
• Mechanical Engineering

Centers, Labs, & Programs

• Abdul Latif Jameel Poverty Action Lab (J-PAL)
• Picower Institute for Learning and Memory
• Lincoln Laboratory
• School of Architecture + Planning
• School of Engineering
• School of Humanities, Arts, and Social Sciences
• Sloan School of Management
• School of Science
• MIT Schwarzman College of Computing

Understanding the visual knowledge of language models

Press contact :.

Previous image Next image

You’ve likely heard that a picture is worth a thousand words, but can a large language model (LLM) get the picture if it’s never seen images before? As it turns out, language models that are trained purely on text have a solid understanding of the visual world. They can write image-rendering code to generate complex scenes with intriguing objects and compositions — and even when that knowledge is not used properly, LLMs can refine their images. Researchers from MIT’s Computer Science and Artificial Intelligence Laboratory (CSAIL) observed this when prompting language models to self-correct their code for different images, where the systems improved on their simple clipart drawings with each query.

The visual knowledge of these language models is gained from how concepts like shapes and colors are described across the internet, whether in language or code. When given a direction like “draw a parrot in the jungle,” users jog the LLM to consider what it’s read in descriptions before. To assess how much visual knowledge LLMs have, the CSAIL team constructed a “vision checkup” for LLMs: using their “Visual Aptitude Dataset,” they tested the models’ abilities to draw, recognize, and self-correct these concepts. Collecting each final draft of these illustrations, the researchers trained a computer vision system that identifies the content of real photos.

“We essentially train a vision system without directly using any visual data,” says Tamar Rott Shaham, co-lead author of the study and an MIT electrical engineering and computer science (EECS) postdoc at CSAIL. “Our team queried language models to write image-rendering codes to generate data for us and then trained the vision system to evaluate natural images. We were inspired by the question of how visual concepts are represented through other mediums, like text. To express their visual knowledge, LLMs can use code as a common ground between text and vision.” To build this dataset, the researchers first queried the models to generate code for different shapes, objects, and scenes. Then, they compiled that code to render simple digital illustrations, like a row of bicycles, showing that LLMs understand spatial relations well enough to draw the two-wheelers in a horizontal row. As another example, the model generated a car-shaped cake, combining two random concepts. The language model also produced a glowing light bulb, indicating its ability to create visual effects.  “Our work shows that when you query an LLM (without multimodal pre-training) to create an image, it knows much more than it seems,” says co-lead author, EECS PhD student, and CSAIL member Pratyusha Sharma. “Let’s say you asked it to draw a chair. The model knows other things about this piece of furniture that it may not have immediately rendered, so users can query the model to improve the visual it produces with each iteration. Surprisingly, the model can iteratively enrich the drawing by improving the rendering code to a significant extent.”

The researchers gathered these illustrations, which were then used to train a computer vision system that can recognize objects within real photos (despite never having seen one before). With this synthetic, text-generated data as its only reference point, the system outperforms other procedurally generated image datasets that were trained with authentic photos. The CSAIL team believes that combining the hidden visual knowledge of LLMs with the artistic capabilities of other AI tools like diffusion models could also be beneficial. Systems like Midjourney sometimes lack the know-how to consistently tweak the finer details in an image, making it difficult for them to handle requests like reducing how many cars are pictured, or placing an object behind another. If an LLM sketched out the requested change for the diffusion model beforehand, the resulting edit could be more satisfactory.

The irony, as Rott Shaham and Sharma acknowledge, is that LLMs sometimes fail to recognize the same concepts that they can draw. This became clear when the models incorrectly identified human re-creations of images within the dataset. Such diverse representations of the visual world likely triggered the language models’ misconceptions. While the models struggled to perceive these abstract depictions, they demonstrated the creativity to draw the same concepts differently each time. When the researchers queried LLMs to draw concepts like strawberries and arcades multiple times, they produced pictures from diverse angles with varying shapes and colors, hinting that the models might have actual mental imagery of visual concepts (rather than reciting examples they saw before).

The CSAIL team believes this procedure could be a baseline for evaluating how well a generative AI model can train a computer vision system. Additionally, the researchers look to expand the tasks they challenge language models on. As for their recent study, the MIT group notes that they don’t have access to the training set of the LLMs they used, making it challenging to further investigate the origin of their visual knowledge. In the future, they intend to explore training an even better vision model by letting the LLM work directly with it.

Sharma and Rott Shaham are joined on the paper by former CSAIL affiliate Stephanie Fu ’22, MNG ’23 and EECS PhD students Manel Baradad, Adrián Rodríguez-Muñoz ’22, and Shivam Duggal, who are all CSAIL affiliates; as well as MIT Associate Professor Phillip Isola and Professor Antonio Torralba. Their work was supported, in part, by a grant from the MIT-IBM Watson AI Lab, a LaCaixa Fellowship, the Zuckerman STEM Leadership Program, and the Viterbi Fellowship. They present their paper this week at the IEEE/CVF Computer Vision and Pattern Recognition Conference.

Share this news article on:

Related links.

• Project website
• Phillip Isola
• Tamar Rott Shaham
• Pratyusha Sharma
• Antonio Torralba
• Computer Science and Artificial Intelligence Laboratory (CSAIL)
• MIT-IBM Watson AI Lab
• Department of Electrical Engineering and Computer Science

Related Topics

• Computer science and technology
• Artificial intelligence
• Computer vision
• Electrical Engineering & Computer Science (eecs)

Related Articles

A simpler path to better computer vision

AI system makes models like DALL-E 2 more creative

Seeing the whole from some of the parts

Rewriting the rules of machine-generated art

Previous item Next item

More MIT News

Faces of MIT: Anthony Hallee-Farrell '13

Read full story →

From group stretches to “Hitting Roman,” MIT Motorsports traditions live on

Creating the crossroads

Study reveals why AI models that analyze medical images can be biased

Leaning into the immune system’s complexity

Scientists use computational modeling to guide a difficult chemical synthesis

• More news on MIT News homepage →

Massachusetts Institute of Technology 77 Massachusetts Avenue, Cambridge, MA, USA

• Map (opens in new window)
• Events (opens in new window)
• People (opens in new window)
• Careers (opens in new window)
• Accessibility
• Social Media Hub
• MIT on Facebook
• MIT on YouTube
• MIT on Instagram

Why Representation Matters and Why It’s Still Not Enough

Reflections on growing up brown, queer, and asian american..

Posted December 27, 2021 | Reviewed by Ekua Hagan

• Positive media representation can be helpful in increasing self-esteem for people of marginalized groups (especially youth).
• Interpersonal contact and exposure through media representation can assist in reducing stereotypes of underrepresented groups.
• Representation in educational curricula and social media can provide validation and support, especially for youth of marginalized groups.

Growing up as a Brown Asian American child of immigrants, I never really saw anyone who looked like me in the media. The TV shows and movies I watched mostly concentrated on blonde-haired, white, or light-skinned protagonists. They also normalized western and heterosexist ideals and behaviors, while hardly ever depicting things that reflected my everyday life. For example, it was equally odd and fascinating that people on TV didn’t eat rice at every meal; that their parents didn’t speak with accents; or that no one seemed to navigate a world of daily microaggressions . Despite these observations, I continued to absorb this mass media—internalizing messages of what my life should be like or what I should aspire to be like.

Because there were so few media images of people who looked like me, I distinctly remember the joy and validation that emerged when I did see those representations. Filipino American actors like Ernie Reyes, Nia Peeples, Dante Basco, and Tia Carrere looked like they could be my cousins. Each time they sporadically appeared in films and television series throughout my youth, their mere presence brought a sense of pride. However, because they never played Filipino characters (e.g., Carrere was Chinese American in Wayne's World ) or their racial identities remained unaddressed (e.g., Basco as Rufio in Hook ), I did not know for certain that they were Filipino American like me. And because the internet was not readily accessible (nor fully informational) until my late adolescence , I could not easily find out.

Through my Ethnic Studies classes as an undergraduate student (and my later research on Asian American and Filipino American experiences with microaggressions), I discovered that my perspectives were not that unique. Many Asian Americans and other people of color often struggle with their racial and ethnic identity development —with many citing how a lack of media representation negatively impacts their self-esteem and overall views of their racial or cultural groups. Scholars and community leaders have declared mottos like how it's "hard to be what you can’t see," asserting that people from marginalized groups do not pursue career or academic opportunities when they are not exposed to such possibilities. For example, when women (and women of color specifically) don’t see themselves represented in STEM fields , they may internalize that such careers are not made for them. When people of color don’t see themselves in the arts or in government positions, they likely learn similar messages too.

Complicating these messages are my intersectional identities as a queer person of color. In my teens, it was heartbreakingly lonely to witness everyday homophobia (especially unnecessary homophobic language) in almost all television programming. The few visual examples I saw of anyone LGBTQ involved mostly white, gay, cisgender people. While there was some comfort in seeing them navigate their coming out processes or overcome heterosexism on screen, their storylines often appeared unrealistic—at least in comparison to the nuanced homophobia I observed in my religious, immigrant family. In some ways, not seeing LGBTQ people of color in the media kept me in the closet for years.

How representation can help

Representation can serve as opportunities for minoritized people to find community support and validation. For example, recent studies have found that social media has given LGBTQ young people the outlets to connect with others—especially when the COVID-19 pandemic has limited in-person opportunities. Given the increased suicidal ideation, depression , and other mental health issues among LGBTQ youth amidst this global pandemic, visibility via social media can possibly save lives. Relatedly, taking Ethnic Studies courses can be valuable in helping students to develop a critical consciousness that is culturally relevant to their lives. In this way, representation can allow students of color to personally connect to school, potentially making their educational pursuits more meaningful.

Further, representation can be helpful in reducing negative stereotypes about other groups. Initially discussed by psychologist Dr. Gordon Allport as Intergroup Contact Theory, researchers believed that the more exposure or contact that people had to groups who were different from them, the less likely they would maintain prejudice . Literature has supported how positive LGBTQ media representation helped transform public opinions about LGBTQ people and their rights. In 2019, the Pew Research Center reported that the general US population significantly changed their views of same-sex marriage in just 15 years—with 60% of the population being opposed in 2004 to 61% in favor in 2019. While there are many other factors that likely influenced these perspective shifts, studies suggest that positive LGBTQ media depictions played a significant role.

For Asian Americans and other groups who have been historically underrepresented in the media, any visibility can feel like a win. For example, Gold House recently featured an article in Vanity Fair , highlighting the power of Asian American visibility in the media—citing blockbuster films like Crazy Rich Asians and Shang-Chi and the Legend of the Ten Rings . Asian American producers like Mindy Kaling of Never Have I Ever and The Sex Lives of College Girls demonstrate how influential creators of color can initiate their own projects and write their own storylines, in order to directly increase representation (and indirectly increase mental health and positive esteem for its audiences of color).

When representation is not enough

However, representation simply is not enough—especially when it is one-dimensional, superficial, or not actually representative. Some scholars describe how Asian American media depictions still tend to reinforce stereotypes, which may negatively impact identity development for Asian American youth. Asian American Studies is still needed to teach about oppression and to combat hate violence. Further, representation might also fail to reflect the true diversity of communities; historically, Brown Asian Americans have been underrepresented in Asian American media, resulting in marginalization within marginalized groups. For example, Filipino Americans—despite being the first Asian American group to settle in the US and one of the largest immigrant groups—remain underrepresented across many sectors, including academia, arts, and government.

Representation should never be the final goal; instead, it should merely be one step toward equity. Having a diverse cast on a television show is meaningless if those storylines promote harmful stereotypes or fail to address societal inequities. Being the “first” at anything is pointless if there aren’t efforts to address the systemic obstacles that prevent people from certain groups from succeeding in the first place.

Instead, representation should be intentional. People in power should aim for their content to reflect their audiences—especially if they know that doing so could assist in increasing people's self-esteem and wellness. People who have the opportunity to represent their identity groups in any sector may make conscious efforts to use their influence to teach (or remind) others that their communities exist. Finally, parents and teachers can be more intentional in ensuring that their children and students always feel seen and validated. By providing youth with visual representations of people they can relate to, they can potentially save future generations from a lifetime of feeling underrepresented or misunderstood.

Kevin Leo Yabut Nadal, Ph.D., is a Distinguished Professor of Psychology at the City University of New York and the author of books including Microaggressions and Traumatic Stress .

• Find a Therapist
• Find a Treatment Center
• Find a Psychiatrist
• Find a Support Group
• Find Online Therapy
• United States
• Brooklyn, NY
• Chicago, IL
• Houston, TX
• Los Angeles, CA
• New York, NY
• Portland, OR
• San Diego, CA
• San Francisco, CA
• Seattle, WA
• Washington, DC
• Asperger's
• Bipolar Disorder
• Chronic Pain
• Eating Disorders
• Passive Aggression
• Personality
• Goal Setting
• Positive Psychology
• Stopping Smoking
• Low Sexual Desire
• Relationships
• Child Development
• Self Tests NEW
• Therapy Center
• Diagnosis Dictionary
• Types of Therapy

At any moment, someone’s aggravating behavior or our own bad luck can set us off on an emotional spiral that could derail our entire day. Here’s how we can face triggers with less reactivity and get on with our lives.

• Emotional Intelligence
• Gaslighting
• Affective Forecasting
• Neuroscience

Initial Thoughts

Perspectives & resources, what is high-quality mathematics instruction and why is it important.

• Page 1: The Importance of High-Quality Mathematics Instruction
• Page 2: A Standards-Based Mathematics Curriculum
• Page 3: Evidence-Based Mathematics Practices

What evidence-based mathematics practices can teachers employ?

• Page 4: Explicit, Systematic Instruction

Page 5: Visual Representations

• Page 6: Schema Instruction
• Page 7: Metacognitive Strategies
• Page 8: Effective Classroom Practices
• Page 9: References & Additional Resources
• Page 10: Credits

Research Shows

• Students who use accurate visual representations are six times more likely to correctly solve mathematics problems than are students who do not use them. However, students who use inaccurate visual representations are less likely to correctly solve mathematics problems than those who do not use visual representations at all. (Boonen, van Wesel, Jolles, & van der Schoot, 2014)
• Students with a learning disability (LD) often do not create accurate visual representations or use them strategically to solve problems. Teaching students to systematically use a visual representation to solve word problems has led to substantial improvements in math achievement for students with learning disabilities. (van Garderen, Scheuermann, & Jackson, 2012; van Garderen, Scheuermann, & Poch, 2014)
• Students who use visual representations to solve word problems are more likely to solve the problems accurately. This was equally true for students who had LD, were low-achieving, or were average-achieving. (Krawec, 2014)

Visual representations are flexible; they can be used across grade levels and types of math problems. They can be used by teachers to teach mathematics facts and by students to learn mathematics content. Visual representations can take a number of forms. Click on the links below to view some of the visual representations most commonly used by teachers and students.

How does this practice align?

High-leverage practice (hlp).

• HLP15 : Provide scaffolded supports

CCSSM: Standards for Mathematical Practice

• MP1 : Make sense of problems and persevere in solving them.

Number Lines

Definition : A straight line that shows the order of and the relation between numbers.

Common Uses : addition, subtraction, counting

Strip Diagrams

Definition : A bar divided into rectangles that accurately represent quantities noted in the problem.

Common Uses : addition, fractions, proportions, ratios

Definition : Simple drawings of concrete or real items (e.g., marbles, trucks).

Common Uses : counting, addition, subtraction, multiplication, division

Graphs/Charts

Definition : Drawings that depict information using lines, shapes, and colors.

Common Uses : comparing numbers, statistics, ratios, algebra

Graphic Organizers

Definition : Visual that assists students in remembering and organizing information, as well as depicting the relationships between ideas (e.g., word webs, tables, Venn diagrams).

Common Uses : algebra, geometry

Before they can solve problems, however, students must first know what type of visual representation to create and use for a given mathematics problem. Some students—specifically, high-achieving students, gifted students—do this automatically, whereas others need to be explicitly taught how. This is especially the case for students who struggle with mathematics and those with mathematics learning disabilities. Without explicit, systematic instruction on how to create and use visual representations, these students often create visual representations that are disorganized or contain incorrect or partial information. Consider the examples below.

Elementary Example

Mrs. Aldridge ask her first-grade students to add 2 + 4 by drawing dots.

Notice that Talia gets the correct answer. However, because Colby draws his dots in haphazard fashion, he fails to count all of them and consequently arrives at the wrong solution.

High School Example

Mr. Huang asks his students to solve the following word problem:

The flagpole needs to be replaced. The school would like to replace it with the same size pole. When Juan stands 11 feet from the base of the pole, the angle of elevation from Juan’s feet to the top of the pole is 70 degrees. How tall is the pole?

Compare the drawings below created by Brody and Zoe to represent this problem. Notice that Brody drew an accurate representation and applied the correct strategy. In contrast, Zoe drew a picture with partially correct information. The 11 is in the correct place, but the 70° is not. As a result of her inaccurate representation, Zoe is unable to move forward and solve the problem. However, given an accurate representation developed by someone else, Zoe is more likely to solve the problem correctly.

Manipulatives

Some students will not be able to grasp mathematics skills and concepts using only the types of visual representations noted in the table above. Very young children and students who struggle with mathematics often require different types of visual representations known as manipulatives. These concrete, hands-on materials and objects—for example, an abacus or coins—help students to represent the mathematical idea they are trying to learn or the problem they are attempting to solve. Manipulatives can help students develop a conceptual understanding of mathematical topics. (For the purpose of this module, the term concrete objects refers to manipulatives and the term visual representations refers to schematic diagrams.)

It is important that the teacher make explicit the connection between the concrete object and the abstract concept being taught. The goal is for the student to eventually understand the concepts and procedures without the use of manipulatives. For secondary students who struggle with mathematics, teachers should show the abstract along with the concrete or visual representation and explicitly make the connection between them.

A move from concrete objects or visual representations to using abstract equations can be difficult for some students. One strategy teachers can use to help students systematically transition among concrete objects, visual representations, and abstract equations is the Concrete-Representational-Abstract (CRA) framework.

If you would like to learn more about this framework, click here.

Concrete-Representational-Abstract Framework

• Concrete —Students interact and manipulate three-dimensional objects, for example algebra tiles or other algebra manipulatives with representations of variables and units.
• Representational — Students use two-dimensional drawings to represent problems. These pictures may be presented to them by the teacher, or through the curriculum used in the class, or students may draw their own representation of the problem.
• Abstract — Students solve problems with numbers, symbols, and words without any concrete or representational assistance.

CRA is effective across all age levels and can assist students in learning concepts, procedures, and applications. When implementing each component, teachers should use explicit, systematic instruction and continually monitor student work to assess their understanding, asking them questions about their thinking and providing clarification as needed. Concrete and representational activities must reflect the actual process of solving the problem so that students are able to generalize the process to solve an abstract equation. The illustration below highlights each of these components.

For Your Information

One promising practice for moving secondary students with mathematics difficulties or disabilities from the use of manipulatives and visual representations to the abstract equation quickly is the CRA-I strategy . In this modified version of CRA, the teacher simultaneously presents the content using concrete objects, visual representations of the concrete objects, and the abstract equation. Studies have shown that this framework is effective for teaching algebra to this population of students (Strickland & Maccini, 2012; Strickland & Maccini, 2013; Strickland, 2017).

Kim Paulsen discusses the benefits of manipulatives and a number of things to keep in mind when using them (time: 2:35).

Kim Paulsen, EdD Associate Professor, Special Education Vanderbilt University

View Transcript

Transcript: Kim Paulsen, EdD

Manipulatives are a great way of helping kids understand conceptually. The use of manipulatives really helps students see that conceptually, and it clicks a little more with them. Some of the things, though, that we need to remember when we’re using manipulatives is that it is important to give students a little bit of free time when you’re using a new manipulative so that they can just explore with them. We need to have specific rules for how to use manipulatives, that they aren’t toys, that they really are learning materials, and how students pick them up, how they put them away, the right time to use them, and making sure that they’re not distracters while we’re actually doing the presentation part of the lesson. One of the important things is that we don’t want students to memorize the algorithm or the procedures while they’re using the manipulatives. It really is just to help them understand conceptually. That doesn’t mean that kids are automatically going to understand conceptually or be able to make that bridge between using the concrete manipulatives into them being able to solve the problems. For some kids, it is difficult to use the manipulatives. That’s not how they learn, and so we don’t want to force kids to have to use manipulatives if it’s not something that is helpful for them. So we have to remember that manipulatives are one way to think about teaching math.

I think part of the reason that some teachers don’t use them is because it takes a lot of time, it takes a lot of organization, and they also feel that students get too reliant on using manipulatives. One way to think about using manipulatives is that you do it a couple of lessons when you’re teaching a new concept, and then take those away so that students are able to do just the computation part of it. It is true we can’t walk around life with manipulatives in our hands. And I think one of the other reasons that a lot of schools or teachers don’t use manipulatives is because they’re very expensive. And so it’s very helpful if all of the teachers in the school can pool resources and have a manipulative room where teachers can go check out manipulatives so that it’s not so expensive. Teachers have to know how to use them, and that takes a lot of practice.

• Business Essentials
• Leadership & Management
• Credential of Leadership, Impact, and Management in Business (CLIMB)
• Entrepreneurship & Innovation
• Digital Transformation
• Finance & Accounting
• Business in Society
• For Organizations
• Support Portal
• Media Coverage
• Founding Donors
• Leadership Team

• Harvard Business School →
• HBS Online →
• Business Insights →

Business Insights

Harvard Business School Online's Business Insights Blog provides the career insights you need to achieve your goals and gain confidence in your business skills.

• Career Development
• Communication
• Decision-Making
• Earning Your MBA
• Negotiation
• News & Events
• Productivity
• Staff Spotlight
• Student Profiles
• Work-Life Balance
• AI Essentials for Business
• Alternative Investments
• Business Analytics
• Business Strategy
• Business and Climate Change
• Design Thinking and Innovation
• Digital Marketing Strategy
• Disruptive Strategy
• Economics for Managers
• Entrepreneurship Essentials
• Financial Accounting
• Global Business
• Launching Tech Ventures
• Leadership Principles
• Leadership, Ethics, and Corporate Accountability
• Leading Change and Organizational Renewal
• Leading with Finance
• Management Essentials
• Negotiation Mastery
• Organizational Leadership
• Power and Influence for Positive Impact
• Strategy Execution
• Sustainable Business Strategy
• Sustainable Investing
• Winning with Digital Platforms

17 Data Visualization Techniques All Professionals Should Know

• 17 Sep 2019

There’s a growing demand for business analytics and data expertise in the workforce. But you don’t need to be a professional analyst to benefit from data-related skills.

Becoming skilled at common data visualization techniques can help you reap the rewards of data-driven decision-making , including increased confidence and potential cost savings. Learning how to effectively visualize data could be the first step toward using data analytics and data science to your advantage to add value to your organization.

Several data visualization techniques can help you become more effective in your role. Here are 17 essential data visualization techniques all professionals should know, as well as tips to help you effectively present your data.

Access your free e-book today.

What Is Data Visualization?

Data visualization is the process of creating graphical representations of information. This process helps the presenter communicate data in a way that’s easy for the viewer to interpret and draw conclusions.

There are many different techniques and tools you can leverage to visualize data, so you want to know which ones to use and when. Here are some of the most important data visualization techniques all professionals should know.

Data Visualization Techniques

The type of data visualization technique you leverage will vary based on the type of data you’re working with, in addition to the story you’re telling with your data .

Here are some important data visualization techniques to know:

• Gantt Chart
• Box and Whisker Plot
• Waterfall Chart
• Scatter Plot
• Pictogram Chart
• Highlight Table
• Bullet Graph
• Choropleth Map
• Network Diagram
• Correlation Matrices

1. Pie Chart

Pie charts are one of the most common and basic data visualization techniques, used across a wide range of applications. Pie charts are ideal for illustrating proportions, or part-to-whole comparisons.

Because pie charts are relatively simple and easy to read, they’re best suited for audiences who might be unfamiliar with the information or are only interested in the key takeaways. For viewers who require a more thorough explanation of the data, pie charts fall short in their ability to display complex information.

2. Bar Chart

The classic bar chart , or bar graph, is another common and easy-to-use method of data visualization. In this type of visualization, one axis of the chart shows the categories being compared, and the other, a measured value. The length of the bar indicates how each group measures according to the value.

One drawback is that labeling and clarity can become problematic when there are too many categories included. Like pie charts, they can also be too simple for more complex data sets.

3. Histogram

Unlike bar charts, histograms illustrate the distribution of data over a continuous interval or defined period. These visualizations are helpful in identifying where values are concentrated, as well as where there are gaps or unusual values.

Histograms are especially useful for showing the frequency of a particular occurrence. For instance, if you’d like to show how many clicks your website received each day over the last week, you can use a histogram. From this visualization, you can quickly determine which days your website saw the greatest and fewest number of clicks.

4. Gantt Chart

Gantt charts are particularly common in project management, as they’re useful in illustrating a project timeline or progression of tasks. In this type of chart, tasks to be performed are listed on the vertical axis and time intervals on the horizontal axis. Horizontal bars in the body of the chart represent the duration of each activity.

Utilizing Gantt charts to display timelines can be incredibly helpful, and enable team members to keep track of every aspect of a project. Even if you’re not a project management professional, familiarizing yourself with Gantt charts can help you stay organized.

5. Heat Map

A heat map is a type of visualization used to show differences in data through variations in color. These charts use color to communicate values in a way that makes it easy for the viewer to quickly identify trends. Having a clear legend is necessary in order for a user to successfully read and interpret a heatmap.

There are many possible applications of heat maps. For example, if you want to analyze which time of day a retail store makes the most sales, you can use a heat map that shows the day of the week on the vertical axis and time of day on the horizontal axis. Then, by shading in the matrix with colors that correspond to the number of sales at each time of day, you can identify trends in the data that allow you to determine the exact times your store experiences the most sales.

6. A Box and Whisker Plot

A box and whisker plot , or box plot, provides a visual summary of data through its quartiles. First, a box is drawn from the first quartile to the third of the data set. A line within the box represents the median. “Whiskers,” or lines, are then drawn extending from the box to the minimum (lower extreme) and maximum (upper extreme). Outliers are represented by individual points that are in-line with the whiskers.

This type of chart is helpful in quickly identifying whether or not the data is symmetrical or skewed, as well as providing a visual summary of the data set that can be easily interpreted.

7. Waterfall Chart

A waterfall chart is a visual representation that illustrates how a value changes as it’s influenced by different factors, such as time. The main goal of this chart is to show the viewer how a value has grown or declined over a defined period. For example, waterfall charts are popular for showing spending or earnings over time.

8. Area Chart

An area chart , or area graph, is a variation on a basic line graph in which the area underneath the line is shaded to represent the total value of each data point. When several data series must be compared on the same graph, stacked area charts are used.

This method of data visualization is useful for showing changes in one or more quantities over time, as well as showing how each quantity combines to make up the whole. Stacked area charts are effective in showing part-to-whole comparisons.

9. Scatter Plot

Another technique commonly used to display data is a scatter plot . A scatter plot displays data for two variables as represented by points plotted against the horizontal and vertical axis. This type of data visualization is useful in illustrating the relationships that exist between variables and can be used to identify trends or correlations in data.

Scatter plots are most effective for fairly large data sets, since it’s often easier to identify trends when there are more data points present. Additionally, the closer the data points are grouped together, the stronger the correlation or trend tends to be.

10. Pictogram Chart

Pictogram charts , or pictograph charts, are particularly useful for presenting simple data in a more visual and engaging way. These charts use icons to visualize data, with each icon representing a different value or category. For example, data about time might be represented by icons of clocks or watches. Each icon can correspond to either a single unit or a set number of units (for example, each icon represents 100 units).

In addition to making the data more engaging, pictogram charts are helpful in situations where language or cultural differences might be a barrier to the audience’s understanding of the data.

11. Timeline

Timelines are the most effective way to visualize a sequence of events in chronological order. They’re typically linear, with key events outlined along the axis. Timelines are used to communicate time-related information and display historical data.

Timelines allow you to highlight the most important events that occurred, or need to occur in the future, and make it easy for the viewer to identify any patterns appearing within the selected time period. While timelines are often relatively simple linear visualizations, they can be made more visually appealing by adding images, colors, fonts, and decorative shapes.

12. Highlight Table

A highlight table is a more engaging alternative to traditional tables. By highlighting cells in the table with color, you can make it easier for viewers to quickly spot trends and patterns in the data. These visualizations are useful for comparing categorical data.

Depending on the data visualization tool you’re using, you may be able to add conditional formatting rules to the table that automatically color cells that meet specified conditions. For instance, when using a highlight table to visualize a company’s sales data, you may color cells red if the sales data is below the goal, or green if sales were above the goal. Unlike a heat map, the colors in a highlight table are discrete and represent a single meaning or value.

13. Bullet Graph

A bullet graph is a variation of a bar graph that can act as an alternative to dashboard gauges to represent performance data. The main use for a bullet graph is to inform the viewer of how a business is performing in comparison to benchmarks that are in place for key business metrics.

In a bullet graph, the darker horizontal bar in the middle of the chart represents the actual value, while the vertical line represents a comparative value, or target. If the horizontal bar passes the vertical line, the target for that metric has been surpassed. Additionally, the segmented colored sections behind the horizontal bar represent range scores, such as “poor,” “fair,” or “good.”

14. Choropleth Maps

A choropleth map uses color, shading, and other patterns to visualize numerical values across geographic regions. These visualizations use a progression of color (or shading) on a spectrum to distinguish high values from low.

Choropleth maps allow viewers to see how a variable changes from one region to the next. A potential downside to this type of visualization is that the exact numerical values aren’t easily accessible because the colors represent a range of values. Some data visualization tools, however, allow you to add interactivity to your map so the exact values are accessible.

15. Word Cloud

A word cloud , or tag cloud, is a visual representation of text data in which the size of the word is proportional to its frequency. The more often a specific word appears in a dataset, the larger it appears in the visualization. In addition to size, words often appear bolder or follow a specific color scheme depending on their frequency.

Word clouds are often used on websites and blogs to identify significant keywords and compare differences in textual data between two sources. They are also useful when analyzing qualitative datasets, such as the specific words consumers used to describe a product.

16. Network Diagram

Network diagrams are a type of data visualization that represent relationships between qualitative data points. These visualizations are composed of nodes and links, also called edges. Nodes are singular data points that are connected to other nodes through edges, which show the relationship between multiple nodes.

There are many use cases for network diagrams, including depicting social networks, highlighting the relationships between employees at an organization, or visualizing product sales across geographic regions.

17. Correlation Matrix

A correlation matrix is a table that shows correlation coefficients between variables. Each cell represents the relationship between two variables, and a color scale is used to communicate whether the variables are correlated and to what extent.

Correlation matrices are useful to summarize and find patterns in large data sets. In business, a correlation matrix might be used to analyze how different data points about a specific product might be related, such as price, advertising spend, launch date, etc.

Other Data Visualization Options

While the examples listed above are some of the most commonly used techniques, there are many other ways you can visualize data to become a more effective communicator. Some other data visualization options include:

• Bubble clouds
• Circle views
• Dendrograms
• Dot distribution maps
• Open-high-low-close charts
• Polar areas
• Radial trees
• Ring Charts
• Sankey diagram
• Span charts
• Streamgraphs
• Wedge stack graphs
• Violin plots

Tips For Creating Effective Visualizations

Creating effective data visualizations requires more than just knowing how to choose the best technique for your needs. There are several considerations you should take into account to maximize your effectiveness when it comes to presenting data.

Related : What to Keep in Mind When Creating Data Visualizations in Excel

One of the most important steps is to evaluate your audience. For example, if you’re presenting financial data to a team that works in an unrelated department, you’ll want to choose a fairly simple illustration. On the other hand, if you’re presenting financial data to a team of finance experts, it’s likely you can safely include more complex information.

Another helpful tip is to avoid unnecessary distractions. Although visual elements like animation can be a great way to add interest, they can also distract from the key points the illustration is trying to convey and hinder the viewer’s ability to quickly understand the information.

Finally, be mindful of the colors you utilize, as well as your overall design. While it’s important that your graphs or charts are visually appealing, there are more practical reasons you might choose one color palette over another. For instance, using low contrast colors can make it difficult for your audience to discern differences between data points. Using colors that are too bold, however, can make the illustration overwhelming or distracting for the viewer.

Related : Bad Data Visualization: 5 Examples of Misleading Data

Visuals to Interpret and Share Information

No matter your role or title within an organization, data visualization is a skill that’s important for all professionals. Being able to effectively present complex data through easy-to-understand visual representations is invaluable when it comes to communicating information with members both inside and outside your business.

There’s no shortage in how data visualization can be applied in the real world. Data is playing an increasingly important role in the marketplace today, and data literacy is the first step in understanding how analytics can be used in business.

Are you interested in improving your analytical skills? Learn more about Business Analytics , our eight-week online course that can help you use data to generate insights and tackle business decisions.

This post was updated on January 20, 2022. It was originally published on September 17, 2019.

About the Author

• SUGGESTED TOPICS
• The Magazine
• Newsletters
• Managing Yourself
• Managing Teams
• Work-life Balance
• The Big Idea
• Data & Visuals
• Reading Lists
• Case Selections
• HBR Learning
• Topic Feeds
• Account Settings
• Email Preferences

Visualizations That Really Work

• Scott Berinato

Not long ago, the ability to create smart data visualizations (or dataviz) was a nice-to-have skill for design- and data-minded managers. But now it’s a must-have skill for all managers, because it’s often the only way to make sense of the work they do. Decision making increasingly relies on data, which arrives with such overwhelming velocity, and in such volume, that some level of abstraction is crucial. Thanks to the internet and a growing number of affordable tools, visualization is accessible for everyone—but that convenience can lead to charts that are merely adequate or even ineffective.

By answering just two questions, Berinato writes, you can set yourself up to succeed: Is the information conceptual or data-driven? and Am I declaring something or exploring something? He leads readers through a simple process of identifying which of the four types of visualization they might use to achieve their goals most effectively: idea illustration, idea generation, visual discovery, or everyday dataviz.

This article is adapted from the author’s just-published book, Good Charts: The HBR Guide to Making Smarter, More Persuasive Data Visualizations.

Know what message you’re trying to communicate before you get down in the weeds.

Idea in Brief

Knowledge workers need greater visual literacy than they used to, because so much data—and so many ideas—are now presented graphically. But few of us have been taught data-visualization skills.

Tools Are Fine…

Inexpensive tools allow anyone to perform simple tasks such as importing spreadsheet data into a bar chart. But that means it’s easy to create terrible charts. Visualization can be so much more: It’s an agile, powerful way to explore ideas and communicate information.

…But Strategy Is Key

Don’t jump straight to execution. Instead, first think about what you’re representing—ideas or data? Then consider your purpose: Do you want to inform, persuade, or explore? The answers will suggest what tools and resources you need.

Not long ago, the ability to create smart data visualizations, or dataviz, was a nice-to-have skill. For the most part, it benefited design- and data-minded managers who made a deliberate decision to invest in acquiring it. That’s changed. Now visual communication is a must-have skill for all managers, because more and more often, it’s the only way to make sense of the work they do.

• Scott Berinato is a senior editor at Harvard Business Review and the author of Good Charts Workbook: Tips Tools, and Exercises for Making Better Data Visualizations and Good Charts: The HBR Guide to Making Smarter, More Persuasive Data Visualizations .

Partner Center

• Reviews / Why join our community?
• For companies
• Frequently asked questions

Guidelines for Good Visual Information Representations

Information visualization is not as easy as it might first appear, particularly when you are examining complex data sets. How do you deliver a “good” representation of the information that you bring out of the data that you are working with?

While this may be a subjective area of information visualization and, of course, there are exceptions to the guidelines (as with all areas of design – rules are for breaking if by breaking them you achieve your purpose) it’s best to begin with the four guidelines outlined by Edward Tufte.

About Edward Tufte

Edward Tufte is, perhaps, the world’s leading authority on information design and data visualization . He is an American statistician and a Professor Emeritus at Yale University (for political sciences, computer sciences and statistics).

He has authored several books and papers on analytic design and is a strong proponent for the power of visualizing data. In particular his books, Visual Display of Quantitative Information, Envisioning Information, Visual Explanations and Beautiful Evidence are considered to be definitive works in the field of information visualization. The New York Times called him; “The Leonardo da Vinci of data.”

Within his works you can find four essential guidelines for visual information representation:

Graphical Excellence

Visual integrity, maximizing the data-ink ratio, aesthetic elegance, tufte’s criteria for good visual information representation.

The purpose of “good’ representations is to deliver a visual representation of data to the user of that representation which is “most fit for purpose”. This will enable the user of the information to make the most out of the representation. There is no single hard and fast rule for creating good representations because the nature of the data, the users of that data, etc. are enormously varied.

Thus we find ourselves with a set of criteria which can be applied to most visual representations, as suggested by Tufte, to judge their fitness for purpose. It must be acknowledged, however, that these criteria can be bent or even broken if doing so serves a purpose for the user of the information representation.

There could be hours of debate as to what constitutes graphical excellence but Tufte offers that in data representations at least it should provide the user with; “the greatest number of ideas, in the shortest time, using the least amount of ink, in the smallest space.”

In short as with many other areas of user experience – the focus here is on usability ; it is completely possible to create beautiful graphical representations of data which fail to deliver on these premises. In fact, it might be said that this occurs so often that the power of data visualization is muted because people have come to expect such visualizations to be decorative rather than valuable.

The graphic above, relating to US employment statistics in March 2015, offers many ideas in a very small space and is easy to digest. We’d suggest it meets the criteria of “graphical excellence”.

This is a confusing term. When Tufte refers to “visual integrity” he is invoking an almost moral position in that the representation should neither distort the underlying data nor create a false impression or interpretation of that data.

In practice this means that numerical scales should be properly proportionate (and not fudged to exaggerate the fall or rise of a curve at a particular point, for example). That variations, when they occur, should relate to the data rather than to the artistic interpretation of that data. The dimensions used within an image should be limited to the dimensions within the data and should never exceed them and finally that the keys (or legends) should be undistorted and unambiguous.

This bar graph fails to give us enough information to be useful and thus fails in delivering “visual integrity”.

Tufte recommends that we pay attention to the way that a visualization is compiled; in that all superfluous elements (to the user) should be removed. He offers the idea that borders, backgrounds, use of 3D, etc. may do nothing but serve to distract the user from the information itself. He promotes that you give priority to the data and how it will be used and not to the visual appearance of that representation.

He also provides a mathematical formula for a data-ink ratio:

Data-Ink/Total Ink Used

This is simply a comparison of the ink needed to clearly and unambiguously present the data to the ink actually used (including aesthetic considerations). The closer the ratio is to 1 – the less distracting your representation is likely to be and thus the more useful it is likely to be for your user.

This image of business processes with an ERP environment is quite good at conveying which business functions are affected by the ERP processes but what purpose does the color scheme serve?

Tufte’s interpretation of aesthetic elegance is not based on the “physical beauty” of an information visualization but rather the simplicity of the design evoking the complexity of the data clearly.

He holds up Minard’s visualization (pictured below) of Napoleon’s March in the Russian Campaign as an example of aesthetic elegance.

The Take Away

Tufte’s guidelines are not prescriptive but rather designed to assist the information visualization professional in creating usable and useful information representations. At their core his rules can be boiled down to keeping things as simple and as honest as possible. The rest simply ensure that you adapt to complexity in the most creative and basic way possible.

UX designers will see clear links between their own design work on products and the design of information representations.

References and Resources

You can find all of Edward Tufte’s work via his website .

Find out more about Charles Joseph Minard and his map of Napoleon’s Russian Campaign.

You can also find an interesting analysis of Minard’s map here .

Hero Image: Author/Copyright holder: Kitware Inc. Copyright terms and licence: CC BY-ND 2.0

UI Design Patterns for Successful Software

Get Weekly Design Tips

Topics in this article, what you should read next, user interface design guidelines: 10 rules of thumb.

• 1.3k shares

Information Overload, Why it Matters and How to Combat It

• 1.1k shares
• 4 years ago

The Key Elements & Principles of Visual Design

How to Develop an Empathic Approach in Design Thinking

• 3 years ago

Simple Guidelines When You Design for Mobile

How to Design an Information Visualization

How to Visualize Your Qualitative User Research Results for Maximum Impact

Preattentive Visual Properties and How to Use Them in Information Visualization

• 5 years ago

6 Common Pitfalls in Prototyping and How to Avoid Them

How to Conduct Focus Groups

Open Access—Link to us!

We believe in Open Access and the  democratization of knowledge . Unfortunately, world-class educational materials such as this page are normally hidden behind paywalls or in expensive textbooks.

If you want this to change , cite this article , link to us, or join us to help us democratize design knowledge !

Privacy Settings

Our digital services use necessary tracking technologies, including third-party cookies, for security, functionality, and to uphold user rights. Optional cookies offer enhanced features, and analytics.

Experience the full potential of our site that remembers your preferences and supports secure sign-in.

Governs the storage of data necessary for maintaining website security, user authentication, and fraud prevention mechanisms.

Enhanced Functionality

Saves your settings and preferences, like your location, for a more personalized experience.

Referral Program

We use cookies to enable our referral program, giving you and your friends discounts.

Error Reporting

We share user ID with Bugsnag and NewRelic to help us track errors and fix issues.

Optimize your experience by allowing us to monitor site usage. You’ll enjoy a smoother, more personalized journey without compromising your privacy.

Analytics Storage

Collects anonymous data on how you navigate and interact, helping us make informed improvements.

Differentiates real visitors from automated bots, ensuring accurate usage data and improving your website experience.

Lets us tailor your digital ads to match your interests, making them more relevant and useful to you.

Advertising Storage

Stores information for better-targeted advertising, enhancing your online ad experience.

Personalization Storage

Permits storing data to personalize content and ads across Google services based on user behavior, enhancing overall user experience.

Advertising Personalization

Allows for content and ad personalization across Google services based on user behavior. This consent enhances user experiences.

Enables personalizing ads based on user data and interactions, allowing for more relevant advertising experiences across Google services.

Receive more relevant advertisements by sharing your interests and behavior with our trusted advertising partners.

Enables better ad targeting and measurement on Meta platforms, making ads you see more relevant.

Allows for improved ad effectiveness and measurement through Meta’s Conversions API, ensuring privacy-compliant data sharing.

LinkedIn Insights

Tracks conversions, retargeting, and web analytics for LinkedIn ad campaigns, enhancing ad relevance and performance.

LinkedIn CAPI

Enhances LinkedIn advertising through server-side event tracking, offering more accurate measurement and personalization.

Google Ads Tag

Tracks ad performance and user engagement, helping deliver ads that are most useful to you.

Share Knowledge, Get Respect!

or copy link

Cite according to academic standards

Simply copy and paste the text below into your bibliographic reference list, onto your blog, or anywhere else. You can also just hyperlink to this article.

New to UX Design? We’re giving you a free ebook!

Download our free ebook The Basics of User Experience Design to learn about core concepts of UX design.

In 9 chapters, we’ll cover: conducting user interviews, design thinking, interaction design, mobile UX design, usability, UX research, and many more!

New to UX Design? We’re Giving You a Free ebook!

• Advanced search

Advanced Search

The Time-Course of Food Representation in the Human Brain

• Find this author on Google Scholar
• Find this author on PubMed
• Search for this author on this site
• ORCID record for Denise Moerel
• ORCID record for Thomas A. Carlson
• Figures & Data
• Info & Metrics

Humans make decisions about food every day. The visual system provides important information that forms a basis for these food decisions. Although previous research has focused on visual object and category representations in the brain, it is still unclear how visually presented food is encoded by the brain. Here, we investigate the time-course of food representations in the brain. We used time-resolved multivariate analyses of electroencephalography (EEG) data, obtained from human participants (both sexes), to determine which food features are represented in the brain and whether focused attention is needed for this. We recorded EEG while participants engaged in two different tasks. In one task, the stimuli were task relevant, whereas in the other task, the stimuli were not task relevant. Our findings indicate that the brain can differentiate between food and nonfood items from ∼112 ms after the stimulus onset. The neural signal at later latencies contained information about food naturalness, how much the food was transformed, as well as the perceived caloric content. This information was present regardless of the task. Information about whether food is immediately ready to eat, however, was only present when the food was task relevant and presented at a slow presentation rate. Furthermore, the recorded brain activity correlated with the behavioral responses in an odd-item-out task. The fast representation of these food features, along with the finding that this information is used to guide food categorization decision-making, suggests that these features are important dimensions along which the representation of foods is organized.

• representational similarity analysis
• visual perception

SfN exclusive license .

Member Log In

Log in using your username and password, purchase access, in this issue.

• Table of Contents
• About the Cover
• Index by author
• Masthead (PDF)

Thank you for sharing this Journal of Neuroscience article.

NOTE: We request your email address only to inform the recipient that it was you who recommended this article, and that it is not junk mail. We do not retain these email addresses.

Citation Manager Formats

• EndNote (tagged)
• EndNote 8 (xml)
• RefWorks Tagged
• Ref Manager

• Tweet Widget
• Facebook Like
• Google Plus One

Jump to section

• Significance Statement
• Introduction
• Materials and Methods
• Data Availability

Responses to this article

Jump to comment:, related articles, cited by..., more in this toc section, research articles.

• Fast inhibition slows and desynchronizes mouse auditory efferent neuron activity
• Symmetry in frontal but not motor and somatosensory cortical projections
• Endoplasmic reticulum and mitochondrial calcium handling dynamically shape slow afterhyperpolarizations in vasopressin magnocellular neurons

Behavioral/Cognitive

The Ultimate Visual Guide to Protein: How Much You Need to Eat Each Day

If you’re struggling to get enough protein this visual guide will help you select which foods to eat each day.

Protein is an essential part of our diets. It's a crucial element in what helps our bodies function properly. Protein helps regulate hormones, transports molecules, acts as an enzyme for chemical reactions and more.

Everyone has different dietary requirements, but for the average person, 100 grams of protein daily is ideal. Keep in mind that if you're active, you may need more protein in your diet.

This visual guide shows what 100 grams of protein look like whether you follow a vegan, vegetarian or omnivore diet. Use it to put your daily protein needs into perspective.

The grams were calculated by taking the information from the nutrition facts label on packaged items and weighing them when necessary. The gram amounts listed in this guide are specific to the products used for this experiment, so your numbers may vary if you look at a different brand of bread or yogurt.

100 grams of protein for omnivores

If you don't have any dietary restrictions, eating 100 grams of protein per day should be pretty easy. Here's one way to do it: 

• Greek yogurt (15 grams of protein) 
• Beef sausage (14 grams)
• 1 ounce of mixed nuts (5 grams)
• Two eggs (12 grams)
• Snack cheese (5 grams)
• Four slices (2 ounces) of deli ham (10 grams)
• Two slices of rye bread (10 grams)
• ½ cup of rolled oats (5 grams)
• One can of tuna (27 grams)

Everything pictured above comes to 103 grams, which puts you slightly over the 100-gram goal. 

100 grams of animal protein

As you can see, getting 100 grams of protein from animal products doesn't take much. This photo shows: 

• Four eggs (24 grams of protein)
• Three beef meatballs (15 grams)
• Two slices (2 ounces) of turkey bacon (10 grams) 
• 3 ounces of turkey breast (24 grams)
• One can of tuna (27 grams) 

This amounts to a perfect 100. If you ate all of this in a day, plus bread and other nonanimal products, you would easily surpass 100 grams of protein in a day. 

100 grams of protein for vegetarians

For vegetarians, 100 grams of protein might look like: 

• Two tablespoons of peanut butter (7 grams)
• One tablespoon of hemp seeds (4 grams) 
• ¼ cup of protein granola (10 grams)
• One scoop of plant-based protein powder (20 grams) 
• Two snack cheeses (10 grams) 
• A single-serve Greek yogurt (15 grams)

This actually comes out to 99 grams of protein, which is pretty close and still a great number to hit for a day. 

100 grams of vegan protein

What you see isn't totally what you get with this photo. In the photo, you see: 

• ¼ cup of protein granola (10 grams of protein)
• One scoop of plant-based protein powder (20 grams)
• 1 ounce of nuts (5 grams)
• Two tablespoons of chia seeds (about 10 grams)
• One tablespoon of hemp seeds (4 grams)
• Two slices of rye bread (10 grams) 
• A protein granola bar (8 grams)
• ½ cup of rolled oats (5 grams) 

This amounts to 79 grams of protein. If we double up on the mixed nuts, chia seeds and hemp seeds, this brings us to 93 grams of protein. You could add an extra tablespoon of peanut butter or eat a full cup of oats, instead of half a cup, to come closer to that 100-gram goal. 

Also, this plate doesn't include any high-protein vegan meat substitutes, such as tofu, tempeh or plant-based meats like the Impossible Burger . Those food sources can make it much easier to get 100 grams of protein than someone who eats a vegan diet . 

Nutrition Guides

• Best Meal Kit Delivery Service
• Best Healthy Meal Delivery Service
• Best Cheap Meal Delivery Service
• Hungryroot Review
• EveryPlate Review
• Best Vegan Meal Delivery Service
• Best Vegetarian Meal Delivery Service
• Best Keto Meal Delivery
• Best Grocery Delivery Service
• Fresh N Lean Review
• Blue Apron vs. Hello Fresh
• Best Weight Loss Programs
• Optavia Diet Review
• Noom Diet Review
• Nutrisystem Diet Review
• Weight Watchers Diet Review
• Noom vs. Weight Watchers
• Best Multivitamins
• Best Multivitamins for Men
• Best Multivitamins for Women
• Best Creatine Supplements
• Best Probiotics
• Best Supplements to Gain Weight
• Best Vitamin Subscription
• Best Vitamins for Energy
• Best Vitamins and Supplements for Joint Health
• Best Vitamins for Healthy Hair, Skin and Nails

Structure transfer and consolidation in visual implicit learning

• Find this author on Google Scholar
• Find this author on PubMed
• Search for this author on this site
• ORCID record for Dominik Garber
• For correspondence: [email protected]
• ORCID record for Jozsef Fiser
• Info/History
• Preview PDF

Transfer learning, the re-application of previously learned higher-level regularities to novel input, is a key challenge in cognition. While previous empirical studies investigated human transfer learning in supervised or reinforcement learning for explicit knowledge, it is unknown whether such transfer occurs during naturally more common implicit and unsupervised learning and if so, how it is related to memory consolidation. We compared the transfer of newly acquired explicit and implicit abstract knowledge during unsupervised learning by extending a visual statistical learning paradigm to a transfer learning context. We found transfer during unsupervised learning but with important differences depending on the explicitness/implicitness of the acquired knowledge. Observers acquiring explicit knowledge during initial learning could transfer the learned structures immediately. In contrast, observers with the same amount but implicit knowledge showed the opposite effect, a structural interference during transfer. However, with sleep between the learning phases, implicit observers switched their behaviour and showed the same pattern of transfer as explicit observers did while still remaining implicit. This effect was specific to sleep and not found after non-sleep consolidation. Our results highlight similarities and differences between explicit and implicit learning while acquiring generalizable higher-level knowledge and relying on consolidation for restructuring internal representations.

Competing Interest Statement

The authors have declared no competing interest.

Supplementary materials added to the manuscript file.

View the discussion thread.

Thank you for your interest in spreading the word about bioRxiv.

NOTE: Your email address is requested solely to identify you as the sender of this article.

Citation Manager Formats

• EndNote (tagged)
• EndNote 8 (xml)
• RefWorks Tagged
• Ref Manager
• Tweet Widget
• Facebook Like
• Google Plus One

Subject Area

• Neuroscience
• Animal Behavior and Cognition (5413)
• Biochemistry (12215)
• Bioengineering (9144)
• Bioinformatics (30154)
• Biophysics (15487)
• Cancer Biology (12586)
• Cell Biology (18073)
• Clinical Trials (138)
• Developmental Biology (9753)
• Ecology (14625)
• Epidemiology (2067)
• Evolutionary Biology (18780)
• Genetics (12551)
• Genomics (17226)
• Immunology (12318)
• Microbiology (29046)
• Molecular Biology (12054)
• Neuroscience (63244)
• Paleontology (463)
• Pathology (1938)
• Pharmacology and Toxicology (3369)
• Physiology (5187)
• Plant Biology (10810)
• Scientific Communication and Education (1710)
• Synthetic Biology (3006)
• Systems Biology (7541)
• Zoology (1692)

Understanding Without Words: Visual Representations in Math, Science and Art

• First Online: 02 November 2021

Cite this chapter

• Kathleen Coessens 5 ,
• Karen François 6 &
• Jean Paul Van Bendegem 7  

Part of the book series: Educational Research ((EDRE,volume 11))

307 Accesses

As knowledge can be condensed in different non-verbal ways of representation, the integration of graphic and visual representations and design in research output helps to expand insight and understanding. Layers of visual charts, maps, diagrams not only aim at synergizing the complexity of a topic with visual simplicity, but also to guide a personal search for and insights into knowledge. However, from research over graphic representation to interpretation and understanding implies a move that is scientific, epistemic, artistic and, last but not least, ethical. This article will consider these four aspects from both the side of the researcher and the receiver/interpreter from three different perspectives. The first perspective will consider the importance of visual representations in science and its recent developments. As a second perspective, we will analyse the discussion concerning the use of diagrams in the philosophy of mathematics. A third perspective will be from an artistic perspective on diagrams, where the visual tells us (sometimes) more than the verbal.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

• Get 10 units per month
• Download Article/Chapter or Ebook
• 1 Unit = 1 Article or 1 Chapter
• Cancel anytime
• Available as PDF
• Read on any device
• Instant download
• Own it forever
• Available as EPUB and PDF
• Compact, lightweight edition
• Dispatched in 3 to 5 business days
• Free shipping worldwide - see info
• Durable hardcover edition

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

Visual Reasoning in Science and Mathematics

Diagrams in Mathematics: On Visual Experience in Peirce

Why Do Mathematicians Need Diagrams? Peirce’s Existential Graphs and the Idea of Immanent Visuality

This is the school typically associated with the mathematician David Hilbert. Although he himself saw formalism as a particular strategy to solve certain specific mathematical questions such as the consistency of arithmetic, nevertheless in the hands mainly of the French Bourbaki group it became an overall philosophy and the famous expression that mathematics is a game of meaningless signs was born. See (Detlefsen, 2005 ).

This seemingly simple graph consisting of 10 vertices and 15 edges is nevertheless of supreme importance in graph theory because of the impressive list of properties it possesses. Starikova ( 2017 ) presents a nice and thorough analysis of the graph (in order to discuss its aesthetic qualities). We just mention that the graph has 120 symmetries.

To be found at http://mathworld.wolfram.com/PetersenGraph.html , consulted Sunday, 17 September 2017.

A famous example is a proof of Augustin Cauchy wherein he made the mistake of inverting the quantifiers. A statement of the form ‘For all x, there is a y such that …’ was interpreted as ‘There is a y, such that for all x …’, which is a stronger statement. It is interesting to mention that this case was already (partially) studied by Imre Lakatos, see (Lakatos, 1976 , Appendix 1), who is often seen as the founding father of the study of mathematical practices.

That being said, the interest in the topic is growing. We just mention (Giaquinto, ), (Manders, ), (Giardino, ) and (Carter, 2010 ) as initiators. Of special interest is the connection that is being made between the philosophical approach and the opportunities offered by cognitive science to study the multiple ways that diagrams can be used an interpreted, see (Mumma & Hamami, 2013 ).

It is interesting that, under the same topic, David Bridges (this volume) develops a similar point of view on arts-based research for education. While Bridges questions the ambiguity of the potential and use of artistic means and expressions as research, we rather consider artistic expressions as enriching methods for knowledge construction, opening new insights by their complexity and layeredness.

Alsina, C., & Nelsen, R. B. (2006). Math made visual . The Mathematical Association of America.

Book   Google Scholar  

Bottici, C. (2014). Imaginal politics: Images beyond the imagination and beyond the imaginary . Columbia University Press.

Carter, J. (2010). Diagrams and proofs in analysis. International Studies in the Philosophy of Science, 24 (1), 1–14.

Article   Google Scholar  

Coessens, K. (2010). Visual praxis: Moving the body, the world and the self. Applied Semiotics/Sémiotique Appliquée, Issue: Translating Culture/Traduire La Culture, 9 (24), 112–143.

Google Scholar  

Detlefsen, M. (2005). Formalism. In S. Shapiro (Ed.), The Oxford handbook of philosophy of mathematics and logic (pp. 236–317). OUP.

Chapter   Google Scholar  

Doruff, S. (2011). Diagrammatic praxis. Journal for Artistic Research , (0). http://www.jar-online.net/diagrammatic-praxis/ . Last accessed 15 Feb 2018.

European Science Foundation (ESF) (2011). The European code of conduct for research integrity . Strasbourg: Ireg

Eisner, E., & Day, D. (2004). Handbook of research and policy in art education . Taylor and Francis Inc.

François, K., & Van Bendegem, J. P. (2010). Ethical-political aspects of statistical literacy. In Proceedings of the ICOTS-8 Eight International Conference on Teaching Statistics , Slovenia, Ljubljana, 11–16 July 2010. https://iase-web.org/documents/papers/icots8/ICOTS8_C258_FRANCOIS.pdf . Last accessed 19 Aug 2017.

Frans, J. (2017). Mathematical explanation. A philosophical analysis of the explanatory value of mathematical proofs. In Unpublished dissertation submitted to fulfil the requirements for the degree of Doctor in Philosophy . Brussels: Vrije Universiteit Brussel.

Gates, P. (2018). The importance of diagrams, graphics and other visual representations in STEM teaching. In R. Jorgensen & K. Larkin (Eds.), STEM education in the junior secondary. The state of play (pp. 169–196). Singapore: Springer.

Giardino, V. (2010). Intuition and visualization in mathematical problem solving. Topoi, 29 (1), 29–39.

Giaquinto, M. (2007). Visual thinking in mathematics . Oxford University Press.

Huff, D. (1954). How to lie with statistics . Norton.

Jones, K. (2012). Connecting research with communities through performative social science. The Qualitative Report, 17 (18), 1–8.

Klee, P. (1972/1953). Pedagogical Sketchbook . New York: Praeger Publishers.

Kostelanetz, R. (1975). Essaying essays: Alternative forms of exposition . Out of London Press.

Lakatos, I. (1976). Proofs and refutations. The logic of mathematical discovery . Cambridge University Press.

Latour, B., & Weibel, P. (Eds.). (2005). Making things public. Atmospheres of democracy . The MIT Press.

Manders, K. (2008). The Euclidean diagram. In P. Mancosu (Ed.), The philosophy of mathematical practice (pp. 80–133). OUP.

Mcniff, S. (2008). Art-based research. In J. Knowles & A. Cole (Eds.), Handbook of the arts in qualitative research: perspectives, methodologies, examples, and issues . California, US: Sage.

Mumma, J., & Hamami, Y. (2013). Prolegomena to a cognitive investigation of euclidean diagrammatic reasoning. Journal of Logic, Language and Information, 22 (4), 421–448.

Nelsen, R. B. (1993). Proofs without words II: More exercises in visual thinking . Mathematical Association of America.

Nelsen, R. B. (2000). Proofs without words: Exercises in visual thinking . Mathematical Association of America.

Neurath, M., & Cohen, R. S. (Eds.). (1973). Otto Neurath: empiricism and sociology. Vienna circle collection (vol. 1). Reidel.

Ranciere, J. (2009). The emancipated spectator , trans. G. Elliott. London: Verso.

Rodda, M. (2014). The diagrammatic spectator. Ephemera—Theory & Politics in Organization, 14 (2), 221–244.

Starikova, I. (2017). Aesthetic preferences in mathematics: A case study. In Philosophia mathematica . https://doi.org/10.1093/philmat/nkx014 . Last accessed 15 Feb 2018.

Stenning, K. (2002). Seeing reason: Image and language in learning to think . OUP.

Suppes, P., Esiner, E., Stanley, J., & Grebbe, M. (1998). The vision thing: Educational research and AERA in the 21st century—part 5: A vision for educational research and AERA in the 21st century. Educational Researcher, 27 (9), 33–35.

Tufte, E. R. (1991). Dequantification in scientific visualization: Is this science or television . Yale University.

Tufte, E. R. (2001a). Envisioning information (1990). Cheshire, CT: Graphics Press.

Tufte, E. R. (2001b). The visual display of quantitative information (1983) (2nd ed.). Graphics Press.

Uebel, T. (2005). Political philosophy of science in logical empiricism. Studies in History and Philosophy of Science., 36 (4), 754–773.

Uebel, T. (2010). What is right about Carnap, Neurath and the left Vienna circle thesis: A refutation. Studies in History and Philosophy of Science, 41 , 214–221.

Wallman, K. (1993). Enhancing statistical literacy: Enriching our society. Journal of the American Statistical Association, 88 (421), 1–8.

Download references

Acknowledgements

Thanks to Joachim Frans (2017) who directed my attention to the work of Nelsen (1993, 2000) in his inspiring Ph.D. thesis on ‘Mathematical explanation’.

Author information

Authors and affiliations.

Logic and Philosophy of Science (CLPS), Vrije Universiteit Brussel, Pleinlaan 2, 1050, Brussel, Belgium

Kathleen Coessens

Vrije Universiteit Brussel, Room 5B425, Pleinlaan 2, B-1050, Brussels, Belgium

Karen François

Vrije Universiteit Brussel, Pleinlaan 2, B-1050, Brussel, Belgium

Jean Paul Van Bendegem

You can also search for this author in PubMed   Google Scholar

Corresponding author

Correspondence to Karen François .

Editor information

Editors and affiliations.

KU Leuven, Leuven, Belgium

Paul Smeyers

Marc Depaepe

Rights and permissions

Reprints and permissions

Copyright information

© 2021 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.

About this chapter

Coessens, K., François, K., Van Bendegem, J.P. (2021). Understanding Without Words: Visual Representations in Math, Science and Art. In: Smeyers, P., Depaepe, M. (eds) Production, Presentation, and Acceleration of Educational Research: Could Less be More?. Educational Research, vol 11. Springer, Singapore. https://doi.org/10.1007/978-981-16-3017-0_9

Download citation

DOI : https://doi.org/10.1007/978-981-16-3017-0_9

Published : 02 November 2021

Publisher Name : Springer, Singapore

Print ISBN : 978-981-16-3016-3

Online ISBN : 978-981-16-3017-0

eBook Packages : Education Education (R0)

Share this chapter

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Publish with us

Policies and ethics

• Find a journal
• Track your research

Tactical Menu

Hair shedding scores: more than heat stress.

Jamie Courter State Beef Genetics Extension Specialist

Traditionally, when the topic of hair shedding arises, it is in the context of mitigating summer heat stress among cattle in the southeastern U.S. while grazing toxic fescue. This is not in error. It is estimated that cattle suffering from fescue toxicosis and heat stress alone cost the beef industry over a billion dollars a year. This makes hair shedding an economically relevant trait in cattle.

As research into hair shedding continues, more information about its importance becomes known. Most recently, a relationship between hair shedding in cattle and their ability to sense change in daylight has been discovered. This suggests that cattle who shed their winter coats earlier are more able to adapt to their environment making hair shedding an indicator trait in cattle regardless of location.

This MU Extension guide is meant to 1) provide background information on the economic importance of hair shedding scores, 2) introduce the hair shedding (1-5) scoring system, 3) discuss how to implement hair shedding into a selection program, and 4) promote hair shedding as a management tool.

Hair Shedding as an Economically Relevant Trait

Anytime a trait can directly be linked to profitability, it is characterized as economically relevant. In the case of hair shedding, cows who shed their winter coat earlier tend to wean heavier calves (Genetic correlation = -0.19). Figure 1 shows the average weaning weight of calves born to dams who began shedding their winter coats between March and July. The average weight of a calf born to a dam that shed in March was 57.2 lbs heavier than those that shed in July.

Most research that investigates the relationship between hair shedding and profitability target weaning weight because it encompasses many different production issues created by heat stress. To start, cows who undergo heat stress in the summer are less likely to get pregnant early in the breeding season. This could be due to low body condition scoring (BCS) because of heat stress while grazing summer pasture before fall breeding, or directly due to heat stress during spring breeding. Regardless, cows bred later in the season also calve later and therefore wean lighter calves.

Secondly, cows who undergo stress after calving may see an impact on milk production, which also impacts weaning weight.

Hair Shedding and Environmental Adaptability

Because increases in temperature happen at the same time hours of daylight are increasing, it is difficult to identify whether animals start shedding due to changes in temperature or daylight. Recent research conducted at the University of Missouri investigated the relationship between the DNA of the animal and temperature 30 days prior to the collection of a hair shedding score. This analysis only identified 17 interactions between DNA and temperature that influenced hair shedding. In a second analysis, researchers instead investigated the interaction between the DNA of the animal and the average length of daylight 30 days before the hair shedding score was observed. This time, there were 1,040 significant DNA-by-daylength associations identified. This supports the idea that cattle shed their winter coats in response to increasing amounts of daylight instead of a drop in temperature. This association is important because it promotes hair shedding as an indicator of an animal’s ability to sense and respond to their environment.

Hair Shedding Scores

What/How: Hair shedding scores represent a visual appraisal of the extent of hair shedding and are reported on a 1 to 5 scale ( Figure 2 ) in which:

• 1: Cattle have shed 100% of their winter coat. All that remains is a shorter, smooth, summer coat.
• 2: Cattle have shed 75% of their winter coat, with a small amount of hair left on their flank and hindquarter.
• 3: Cattle have shed approximately 50% of their winter coat. In addition to the hair along the neck, this will include hair along the body, often in patchy spots.
• 4: Cattle have shed only 25% of their winter coat. This will mainly occur around their neck but may also include their topline.
• 5: Cattle have shed 0% of their winter coat. Thick, longer hair still covers their entire body.

Half scores, such as 3.5, are not reported. In general, cattle tend to shed hair from the front to the back and from their topline to their belly ( Figure 3 ), but there is individual animal variation in this pattern. Typically, animals begin shedding around their neck, followed by their topline. The last spots to shed are an animal’s lower quarter above its hock and its underline.

When: It is only necessary to collect hair shedding scores once in late spring or early summer. The date to evaluate cattle for shedding progress will vary by geographic location and environmental conditions. The goal should be to score cattle when there is the most variation in hair shedding within a herd. In other words, a few animals with a hair shedding score of 1 or 5 with a majority receiving a hair shedding score of 3. Mid-May has been identified as an ideal hair shedding evaluation period for cattle in the Southeastern U.S. As a rule of thumb, the hotter and more humid the climate the earlier in the spring scores should be collected.

Who: If using hair shedding score as a selection method or reporting scores to a breed association, all cows in the herd should be observed. While it is recommended to score all animals in a herd on the same day, it is important to keep in mind that males tend to begin shedding a few weeks prior to females and therefore should likely be scored separately.

Where: Being a subjective observation of the amount of hair an animal has shed, these scores are easy to collect. This can be done as cattle pass through a chute during routine handling timeframes or while out on pasture.

Methods of Selecting for Hair Shedding

As a moderately heritable trait (h2 = 0.35 to 0.42), producers can expect to create positive genetic change in their herds by simply adding hair shedding scores as a selection criterion when making selection decisions. To do this, producers would need to assess the hair shedding score of the whole herd on the same day, consider culling older animals with higher scores (more hair), and selecting the replacement heifers who shed earlier in the season in addition to other components of interest.

In addition to phenotypic selection, some bulls will also have an expected progeny difference (EPD) for hair shedding available for use. If available, selecting bulls with a lower hair shedding EPD will result in daughters born who shed earlier in the season, on average.

Hair Shedding and Age

When using hair shedding as a selection criterion, it is important to also consider the age and nutrient requirements of the female. Yearlings and first-calf heifers tend to have higher hair shedding scores compared to older, established cows ( Figure 4 ). This does not mean younger females are necessarily worse shedders than their dams. Younger cows are, by default, the most nutritionally stressed as they are growing and raising a calf while also growing themselves. Therefore, it may be more beneficial to rank and select younger females within their age group instead of comparing them to older herd mates.

When evaluating the effect of age on hair shedding score, the average score for each age group tends to decrease as age increases ( Figure 4 ). This could reflect the impact of late shedding on production. Cows who shed their winter coats later in the summer may have fallen out of the herd due to weaning lighter calves, failure to conceive, or low body condition.

Hair Shedding as a Management Tool

It is anticipated that hair shedding scores could be used in conjunction with body condition scores to assess the current nutritional stress of the herd. Genetic associations were also discovered between hair shedding and functions related to metabolism. Therefore, hair shedding may also pose as an indication of an animal’s overall nutritional plane, thus helping to inform management decisions. It is no coincidence that younger females shed their coats later than their older herd mates. Similarly, older cows who may have had a ‘hard winter’ would shed later in the year. The repeatability of hair shedding is only 45%, which indicates over half (55%) of the variance in hair shedding is due to year-to-year differences in management and environment of the cow. Understanding that later hair shedding (higher scores) indicates increased nutritional demands could be used to identify animals who would benefit from additional supplemental feed heading into spring and summer. 

Although hair shedding has traditionally been associated with heat stress and fescue toxicosis, recent research shows this quick and easy phenotypic assessment of cattle could be a trait of even more economic importance. Producers wishing to select females based on hair shedding scores can do so based on a simple 1 to 5 scoring system. With its moderate heritability, combining this score with a hair shedding EPD or score on bulls would result in positive genetic progress over time.

More detailed information on the scoring system and some frequently asked questions can be found in publication G2041, How to Use the Hair Shedding Scale .

Related publications

• Hair Shedding Scores: A Tool to Select Heat Tolerant Cattle
• How to Use the Hair Shedding Scale