What do we touch when we visualize data?

 

Michael R. Doyle, Laboratory for Environmental and Urban Economics (LEURE), EPFL

For the special call Visual(izing) Data

What do we see when we visualize data? Why choose to interpret and communicate data through any medium other than text? An image is worth a thousand words—but whose words? Who or what bestows meaning upon the image to which text alone cannot do justice? Media theory suggests that visualization is more than meets the eye. From the translation of voice into alphabets to the recording of sound on wax, to the chemical transformations on paper by light exposure, media allow us to externalize sensory impressions and hold them at a distance (Kittler, 1999; McLuhan, 1994). They give us the impression that the world is somehow ‘out there’, that media give us a privileged access to the world that surrounds us—as if the visualization of information establishes the factuality of the data our various apparatuses of investigation uncover. However, if much of today’s media began as measuring devices (e.g. experimentations with electromagnetic wave propagation were precursors to the radio, see Ernst, 2013), and if quantum physics suggests that our understanding of phenomena in the world is inseparable from the instruments used to measure them (Barad, 2007), then the visualization of data—the mediation/measurement of data/information is of particular interest. We are entangled in the agencies of measurement—as part of the phenomena of media—and are in touch with the phenomena that we seek to hold at a distance.

So the question is not only visual, it is also haptic:
What do we touch when we visualize data?

Ratios establish relationships. Numbers extend our sense of touch and establish a relational logic (ratio/nalization) within measured phenomena (McLuhan, 1994). To visualize these relationships places them at a distance and renders them available for further investigation, for public discourse. Such relationships may be known in advance and the researcher seeks to account for the marks left—paths traced in perpetual comings and goings—on, for example, the sea (Figure 1) or the spaces between and through buildings (Figure 2). When it is the data itself that reveals such relationships (e.g. between urban forms whose morphologies are the result of algorithms, Figure 3), the visualization puts the observer (the agent of measurement) back in touch with the phenomenon of interest.

Figure 1. Urban Hardware. Couling, 2015

Figure 1. Urban Hardware. Couling, 2015

Figure 2. Atlas. Danalet, 2015

Figure 2. Atlas. Danalet, 2015

Figure 3. Qinglonghuzhen, Peri-Urban Beijing. Patt, 2015

Figure 3. Qinglonghuzhen, Peri-Urban Beijing. Patt, 2015

But wait—is not visualization only a discretization, a breaking down of phenomena into their component parts? And is not their visualization the ultimate disembodiment of our sense faculties—the setting at a distance that permits us to say ‘look what I found’? In this sense, would it not actually be a breaking apart, an un-touching, a ‘do not touch’ sign that ensures the phenomenon’s stability? There is perhaps something stabilizing, reassuring, in the act of measurement. But if media are productive of phenomena as Mark Hansen (2014) argues, then visualization as a form of touch is in no way passive. It is participatory.

According to physicist Karen Barad (2014), the nature of touch is in fact the inability to touch. Physics explains touch as the repelling of electrons, which quantum physics expands to the indeterminate possibility of touch, an “infinite alterity” (158). The relationship established by ratios between elements (and within phenomena) is a moment of touch in a particular logic of holding apart. And the visualization of these relationships as information (as data), in seemingly setting us apart from them establishes our relationship with them, bringing us back in touch. Generative algorithms (Figure 3), underground potential (Figure 4) mean little until we see and situate ourselves within their ratios, the particular numerical relationships that hold together and apart. Even the eye itself leaves marks that can be traced, aggregated and situated (Figure 5, forthcoming) as if we are invited to walk the visual wish paths of former passersby. It is exciting to touch in this way—“intoxication is a number” (McLuhan citing Baudelaire, 109).

Figure 4. Dig Deep. Doyle, 2015

Figure 4. Dig Deep. Doyle, 2015

The early experiments with moving pictures used slow motion to study human movement in order to render it more efficient, to master it and eventually to automatize it (Kittler, 1999). The visualization of data in turn sets phenomena in time, through an articulation of different time-critical media (Ernst, 2013). It renders the time of phenomena apparent in both its extensions, such as the slowing down of and compilation of the movement of mice (Figure 6, where the mouse may actually be an effort to ethically touch the human), as well as in its contractions, such as the aggregation of the movement and change of light-waves-in-time (Figure 7) or of the tourist snapshots of landmarks and neighborhoods in Manhattan (Figure 8). Visualizing data establishes time as something embedded within phenomena themselves—not as something entirely predetermined—but rather it “transforms these physical processes into time which henceforth cannot exist outside of media operationality” (Hansen, 4). To touch the physical process through its measurement and articulation is to touch time itself.

Figure 6. Intact, Acute, Chronic. Vollenweider, 2015

Figure 6. Intact, Acute, Chronic. Vollenweider, 2015

Figure 7. Cumulative Contrast. Rockcastle, 2015

Figure 7. Cumulative Contrast. Rockcastle, 2015

Figure 8. 3D Attractiveness of New York. Paldino & Sobolevsky, 2015

Figure 8. 3D Attractiveness of New York. Paldino & Sobolevsky, 2015

Shifting the focus from the visualization of data to the touching of phenomena reveals the problematic nature of the photograph as a visualization of data/information. In fact, no photographs were submitted for this call and only one addresses light (Figure 7). Photography as a measurement of light captures an optical moment that is meant, by imitating the eye as an apparatus, to put us in touch with a simulacrum of the real (‘photorealism’) that betrays the distance between observation and the agencies of the observer. The photo-as-touch is too raw in our minds; the information is too entangled with the camera, which was held in the hands of an intentional being whose ulterior motives are unknown. So the quantification and situated measurement of light intensities in space and time walks a delicate line by establishing a mathematical relationship between quantities of light and chromatics that put us in touch with the luminosity of the world. We comfort ourselves in thinking we have kept the phenomena at arms length.

The tendency for the information contained in a photograph to potentially touch us in ways that make us uncomfortable reveals the normative nature of our practices of data visualization. The explanation of the mechanisms of normativity often uses the example of games and it is therefore quite fortuitous that one of the submissions to this call would have visualized professional sports statistics (Figure 9). In Understanding Media, Marshall McLuhan (1994) reminds us that games as a specific form of mass media engage the spectator in participation, in a role that is outside that of the routines of everyday life and provides an almost emancipatory feeling. But the optical bias of the word spectator (from Latin spectare, to ‘observe’) is deceiving. Indeed the spectators fill an important role that ensures that the game remains just that (and betray it when they enter the field in mass protest). But the collection and interest by the ‘spectators’ in the statistics of the game brings these ‘observers’ closer to the players. Play is quantified, set in relationships that are rendered visually. The number of times a player’s hand meets the ball is a touch. Counting these meetings of ball and hand, tracking their accumulations, places the audience in touch with touches—with the players. It extends their touch beyond what is sanctioned in the arena.

Figure 9. Soulmates. Becker, 2015

Figure 9. Soulmates. Becker, 2015

In visualizing data, in touching phenomena in particular ways, researchers consciously or unconsciously follow a set of rules, by which they must abide if they are to commit to the normative structure of their community of peers (whether institutional or disciplinary or otherwise). Of course science is no game, as Joseph Rouse in How Scientific Practices Matter (2002) reminds us, because in scientific practices the rules themselves are in constant question. Part of this questioning seeks to establish an objectivity that is a disinterested gaze. But if the act of visualizing data is a form of touch, then this contact entails a response and a response-ability towards the phenomena we touch (Barad, 2014) and towards the ‘distance’ the rules of the game establish within the measuring media that actualize them. To work within this entanglement is an opportunity not an obstacle, but should not be tread upon—touched upon—lightly. Visualizing data is no spectator sport.

Acknowledgments

This essay was originally written for the PhD colloquy An Untimely Nature of Communication: The Cyphered Reality of Channels and the Birth of Geometry in Encryption and Deciphering—Towards a Physics of Communication given by the Chair for CAAD at the ETH in Zürich. The author would like to thank the instructor Dr. Vera Bühlmann for her comments and insight as well as the authors of the visualizations for submitting to the Visual(izing) Data call.

Cited works

Barad, K. (2014). On Touching — The Inhuman That Therefore I Am (v1.1). In K. Stakemeier & S. Witzgall (Eds.), Power of Material/Politics of Materiality (pp. 153–164). Zurich: diaphanes.
Barad, K. M. (2007). Meeting the universe halfway: quantum physics and the entanglement of matter and meaning. Durham: Duke University Press.
Ernst, W. (2013). Digital memory and the archive. Minneapolis, MN: University of Minnesota Press.
Hansen, M. B. N. (2014). Entangled in Media. Towards a Speculative Phenomenology of Microtemporal Operations. In Philosophy after Nature: Joint Annual Conference of the Society for European Philosophy and the Forum for European Philosophy. Utrecht University. Unpublished.
Kittler, F. A. (1999). Gramophone, film, typewriter. Stanford, Calif: Stanford University Press.
McLuhan, M. (1994). Understanding media: the extensions of man (1st MIT Press ed). Cambridge, Mass: MIT Press.
Rouse, J. (2002). How scientific practices matter: reclaiming philosophical naturalism. Chicago: University of Chicago Press.