As a researcher, my focus is on critically examining digital literature—texts and writing practices where digital media is rendered integral to the experience of literary or poetic forms. In recent years, I have been working so that my research manifests not just at the level of written critique, but also through creative practice, placing my scholarship into a concrete dialogue with the very technologies and techniques under scrutiny. The resulting artworks represent not simply expressions of creative intent, but are vehicles for my investigations into digital literature more broadly.

To illustrate, one such project is Waveform (2017-ongoing), in which an airborne camera drone is deployed as an apparatus for generating poetry. This project represents a practice-led enquiry into what forms might be taken by digital writing in response to modern ecological concerns (see Carter 2017).

Another project, which will be the focus of discussion here, is an ongoing series of works entitled Solid State Poetry. In working on this project since 2014, I have been meditating on how digital inscription is perceived and interpreted by both human and machinic observers, employing a mode of speculative visualisation in which poetic text is rendered into kaleidoscopic patterns. In the earliest works, this process was conducted entirely by hand, but currently both the source poems and the final images are produced using a custom software routine, developed using the open-source Processing toolkit.

Described succinctly, the encoding practices behind Solid State Poetry involve generating a new poem algorithmically using simple cut-up techniques on a source vocabulary, before analysing its verbal structures (such as meter, line, and word-count), and then inscribing these values as a series of transformations on a matrix of Truchet tiles. The final image, therefore, does not encode the source poem word-for-word, and neither is it revealed subsequently in a separate statement. The viewer is thus left free to meditate on what the visual pattern might express with regards to the poetic inscription that generated it—to effectively envision their own poetic source.

Some typical outputs from this process are illustrated below.

Source: Carter 2018.

A key point to reiterate is that the source poems operate as a signal generator for the patterns that follow out from them, and it is the conversion process itself that constitutes the conceptual nexus of Solid State Poetry. This ‘black box’ approach is reflected in the name of the series, with the poems treated as a matrix of potentiality—a representation of poetic expression more broadly—as opposed to a stream of discrete outputs. The reasoning behind this choice will become clearer shortly, but so as to preserve the viewing relations described above, little else will be said regarding the actual poetic content of the series.

To begin unpacking the conceptual premise of Solid State Poetry, it can be noted that the specific choice of poetry as a subject was made in-light of comparing its density of expression with the encoding of digital information. Poetry, as one of the most sophisticated modes of writing available to human actors, radiant with potential readings, appears to have little connection with schemas designed to encode electrical signals for transmission across noisy channels. Both certainly follow very different imperatives, with one seeking to reduce the potential for novelty, for error, whereas the other is seeking to generate new modalities of thought and expression. Nevertheless, both represent a specific means of compressing and reworking the normative vectors of written language, albeit for very different audiences, human and machinic. Thus, while the operations of electronic signal processing manifest far below the thresholds of the human sensorium (hinted only transiently in the form of glitches and errors), it might be speculated how, if it could be rendered observable, it might be treatable as another, nonhuman mode of poetic performance—one whose abstract qualities would allude towards the mechanisms through which information is captured and conveyed within digital systems, and so inspire a rethinking about what they represent as expressive agents.

To rephrase the above in more compact terms: by remediating its poetic source into a structured matrix of abstract shapes, Solid State Poetry acts as a speculative visualisation of the modes by which digital systems prehend the messages of their human users—depicting these systems not simply as utilitarian conveyers of information, but as specific actors with their own expressive vectors.

To help illustrate what we might make of this notion of treating digital systems as having their own stake in interpreting and performing written text, I have found it instructive to reflect on the abstract patterns yielded by Solid State Poetry in light of those found in the work of Anni Albers, the foremost Western textile artist of the 20th Century.

Albers’s beautiful and highly sophisticated art is characterised by its meditation on the emergent relations between text as code and textiles as both a progenitor and realisation of this code. Troy (1999: 28) recounts how, over the course of a prolific career, Albers demonstrated ‘thread as a carrier of meaning, not simply as a utilitarian product’ (emphasis original), and describes her varied engagements with material mark-making:

[Albers] embedded her work with poetic content by exploring in thread the notion of the pictograph (a sign or mark that refers to an external subject), the calligraph (a beautiful mark that stands for a letter or word), and the ideograph (a sign that indicates an idea, not necessarily through pictorial representation).

An illustration of this approach can be found in her piece Ancient Writing (1936). Inspired profoundly by the work of ancient Peruvian weavers, Albers enacted a style that ‘evoked the idea of visual language by grouping together differently textured and patterned squares like words or glyphs, locking this “text” into an underlying grid. The “text,” which is set within margins, appears to jump forward to be “read” like words on a page’ (Troy 31).

Even more significant than these visual effects, however, were the actual means through which they emerged. Troy (28) recounts how Albers’s systematic approach to textile production can be characterised as the application of modular rules, a code, but one that was only apparent in the actual practice of making the textile itself—a flowing, experimental composition that marked a distinctive break from the European tradition of realising a pre-existing design. Troy (30) observes subsequently how the patterns resulting were primarily self-referential, though suggestive also of ‘the image and the idea of text’, leaving the viewer to scan ‘the images for clues to a code, and by doing so [becoming] engaged in a perceptual activity not unlike that of reading’.

What is manifest in Albers’s art, then, is its intrinsic performativity, in that it is the emergent outcome of a set of modular rules, and with symbolic or ‘text-like’ attributes that can be ‘read’ subsequently as tracing the narrative thread of the weaving process. In short, we are encouraged to engage her pieces not as representing a set of underlying codes, but as an expression of the concrete processes they set into motion, and which constitute their chief conceptual charge.

At this point, it is interesting to contrast Albers’s practice with preceding innovations in mechanical handweaving in the early 19th century. In 1804, Joseph Marie Jacquard developed a control system for mechanical handlooms, the Jacquard head, which became one of the first, widespread applications of programmable mechanism. Representing a refinement of earlier systems by various inventors, including Basile Bouchon and Jacques de Vaucanson, the Jacquard head enabled the automatic weaving of almost any visual representation imaginable, so long as it could be encoded into a series of punched cards.

Source: Author, via the London Science Museum.

Arriving well over a century before the advent of electronic storage, the punched cards of the Jacquard head were a precursor to similar systems used in early computers for the task of storing and executing instructions. Effectively, the abstract groupings of punched holes became its own mode of writing, but one that was chiefly readable by a machine.

At one level, Jacquard’s invention was a merging of the European weaving tradition with the technologies and mindset that would later underpin the Industrial Revolution—of a world reducible to symbolic, rationally ordered forms that could provide an idealised template for practice. This approach contrasts with Albers’s use of codes not as representations to be followed, but as catalysts for the performance of weaving itself

Nevertheless, it remains worthwhile to emphasise how Albers’s modular rules and Jacquard’s punched cards were both a means of parsing codes into practice, via the expressive capacities of the mechanical loom. While these codes, taken as a whole, may still be viewable as expressing an abstract concept or a predefined image, at the level of the loom itself they manifest only as specific prompts for action, and thus even Jacquard’s cards subordinate the representational for the performative—of control rods sliding into punched spacings, altering the flow of thread.

When it comes to digital technologies, we can observe a similar collapsing of the representational within the domain of digital inscription, whereby the written message is parsed into a set of executable instructions that animate the circuitry of which they are part, becoming thereafter their chief mode of significance. These instructions do not exist nor operate in isolation, but are components of myriad, interlocking, and continually actioned matrices of code. The viewable digital text cannot be posited therefore as a discrete, isolated element, but as an assemblage that is enacted anew with every processing cycle. As variables, inputs, and conditions change, this assemblage reconfigures itself to accept these changes. Therefore, while a broad continuity may still be evident at the level of the final user output, the underlying operational behaviour can be described in near-emergent terms, for the myriad configurations adopted cannot be accounted for specifically, only accommodated contingently by the system. Hayles (2006: 181-2) summarises this situation neatly, as it relates to digital poetry:

In digital media, the poem has a distributed existence spread among data files and commands, software that executes the commands, and hardware on which the software runs. These digital characteristics imply that the poem ceases to exist as a self-contained object and instead becomes a process, an event brought into existence when the program runs on the appropriate software loaded onto the right hardware. The poem is “eventilized,” made more an event and less a discrete, self-contained object with clear boundaries in space and time.

Hayles (2008:45) has written subsequently on how all acts of digital writing are but an aspect of this greater performance of enacted codes that we call digital computing, and observes this spanning across the thresholds ostensibly separating the biological, the cultural, and the technological, what she terms ‘intermediation’. Nevertheless, to focus on the particular agency of the latter, as a nonhuman actor in the world, emphasises especially its emergent, performative qualities at every stage of recording, parsing, and displaying the messages of its human users, and doing so in ways that are often treated as unremarkable as they are unobservable—being ‘black boxed’ behind smoothly obedient, ‘intuitive’ interfaces, and, in their actual operation, exceeding the scale and duration of human sensory prehension.

Writing practices that integrate digital multimedia can help draw our attention to these underlying processes, bringing to the surface a sense of the condition of digital text as both human readable and as executable code—as components of a machinic performance. In the case of Solid State Poetry, this aim is sought by visualising the parsing of written messages into the structured numerical patterns of digital logic, with their actual semantic content being neither perceived nor acted upon. While the mathematical breakdown of written messages may still be interpretable in conceptual terms, such schemas are, nevertheless, intrinsic not to the human scale of reading, but to the prehension of digital systems, where they operate as catalysts for the execution of algorithms. Therefore, while the patterns generated by Solid State Poetry are indeed so abstract as to convey very little of their source, they give subsequent cause for reflecting on the broader performances by which machinic actors read and articulate the contemporary environment—a hinting of the machinic umwelt.

Extant media coverage of smart devices, bots, and artificial intelligence would suggest readily that the condition of living amongst machinic agents is a recent development, but paying closer attention to the functioning of all digital systems, and their cultural and historical antecedents, reveals a far more complex and longstanding picture. Art and literature, digital or otherwise, represent important vehicles for bringing this richer narrative to light. While it remains a modest gesture, my hope is that as Solid State Poetry grows and develops, alongside my own critical investigations, it will continue to provide a catalyst for further such enquiry and reflection.

Interested readers can follow the development of Solid State Poetry at the writer’s website. A printed collection of selected outputs is soon to be made available.

References:

Carter, R. (2017) ‘Waveform’. The Writing Platform. Available at: http://thewritingplatform.com/2017/09/drone-poetry-deploying-sensory-technologies-tools-writing/ [Accessed 28 August 2018].

Carter, R. (2018) ‘Solid State Poetry’. Available at: http://richardacarter.com/solid-state-poetry/ [Accessed 28 August 2018].

Hayles, N. (2006) ‘The Time of Digital Poetry: From Object to Event’. In: Morris, A., and Swiss, T. (eds.) New Media Poetics: Contexts, Technotexts, and Theories. Cambridge.M.A.: MIT Press, pp. 181-209.

Hayles, N. (2008) Electronic Literature: New Horizons for the Literary. Notre Dame, IN.: Notre Dame Press.

Troy, V. (1999) ‘Thread as Text: The Woven Work of Anni Albers’. In Weber, N. and Asbaghi, P. (eds.)

In the Summer of 2017, I have found myself in the novel position of teaching a drone to write poetry. Such endeavours invariably have a story behind them, and this article will offer a short background on how my practice-led approach towards research has resulted in this latest project—which, at a still nascent stage of development, is entitled Waveform.

My focus, both as an artist and researcher, is on generating different methods through which digital media, as a diverse matrix of activities, artefacts, and infrastructures, can be investigated, explicated, and presented anew. In recent years, I have sought in-particular to engage and evaluate the formidable complexities of the contemporary digital environment from a practitioner’s standpoint—to treat it as a “structure for what does not yet exist”, to appropriate a phrase from pioneering digital author Michael Joyce (2003, p. 616), and explore what potential modes of thought and expression it might support.

My approach is informed by what has been characterised as the ‘performative’, ‘materialist’ turn of the past decade in the humanities. Here, it is understood that human knowledge-making operates not just at the level of the textual, the abstract, and the ocular, but is entangled fundamentally with the entire range of material structures and processes that both constitute and render the world observable (see Salter, 2010). This outlook emphasises how the world’s unfolding takes place at scales and durations far beyond our own native capacities of sensing and knowing—that there is always an excess of possible perspectives, sensations, and relations, which can never be captured fully.

From this standpoint, the production of knowledge always involves an exploratory, creative aspect, with the world being performed anew within the space of every enquiry, actualising yet more of its potential. Moreover, such performances necessitate the entangling of different actors, both human and nonhuman, in modulating their varying capacities of perception and action within the experimental milieu they constitute—actualising this in such a way as to specify and then measure certain processes involved. We might think of scientific equipment here, in which complex and varied apparatuses, tuned assemblies of different energies and materials, perform specific phenomena in ways that can be registered and rendered measurable from the standpoint of human perceptual and cognitive thresholds—but which, importantly, never exhaust the possible behaviours, measurements, and understandings that might be made subsequently (see Pickering, 1995; Barad, 2003). Another example can be found in the creation of new artistic works. Here, the properties of different mediums, tools, and techniques are engaged and explored through practice, establishing a dynamic whose outcome represents the crystallised trace of this exchange between human and material agency, with the impressions of unrealised possibilities being the catalyst for future work.

To understand the resonance of these critical sentiments for the study of digital technology specifically, we can observe statements made in the context of media theory and practice by writers such as Jussi Parikka (2012) who discusses how “media critique is not only about saying things, it is about design and materiality—doing critique in an alternative fashion, against the grain, so to speak” (p. 43). Here, the production of experimental, speculative, or ‘imaginary’ digital media functions not as an abstract window onto an a priori domain, but as an apparatus for entangling critical thought within the concrete structures, practices, and assumptions constituting the contemporary environment, enacting the conditions in which novel insights and alternatives might emerge.

It is visions such as these which have informed my efforts to place practice-led methodologies at the heart of my own research. Interested as I was in the creative entanglement of literary expression with the capabilities of digital multimedia—often referenced as ‘digital’ or ‘electronic literature’—this drive manifested initially in the production of what I termed ‘Solid State Poetry’ (see Carter, 2017). This involved the parsing of text into visual forms that would illustrate how digital systems, as formations that embed and sustain discrete information environments, possess a very different quality of perceptual agency from that of the human—rendering the slippages, ambiguities, and fluidity of our written communications into numerically encoded electrical signals, whose advantages stem precisely from the way they exceed the spatial and temporal thresholds by which we come to know and express the world.

It was in developing this basic agenda that I considered how I might invert these initial predicates and discover ways of parsing visual data—digital images—into a form of poetry. Such efforts would provide a catalyst for reflecting in more detail not just on the different modes by which human beings and digital systems sense and make sense of the world, but to present a vehicle for considering the material conditions in which digital sensing is made possible in the first instance—the specific hardware, imaging software, and signal extraction techniques involved. Moreover, given the popular association of digital data gathering with ever-increasing degrees of pervasiveness, resolution, and precision, an opportunity would be presented to deconstruct the genealogies of technology and discourse that enable and perpetuate these visions, outlining the basis for an alternative, more contingent depiction of the sensory in the process.

It was in pursuing these lines of enquiry that I assembled the basic components of my latest project, Waveform—a title which evokes the processing of continuous environmental signals into the discrete ‘waveforms’ utilised by digital systems. Here, a drone platform is being used to capture images of different patterns of incoming ocean waves across various Cornish beaches. This choice of location is driven by my interest in the role of digital sensors in the gathering of environmental data—part of an increasingly urgent scientific drive to map the impact of human agency at a global scale. Furthermore, in using a drone, as opposed to a conventional digital camera, I have sought to present a more explicit instance of an observing agent operating as part of a wider sensory and interpretative network, and so undercutting any notion of sensory systems as presenting a Cartesian ‘view from nowhere’.

The images gathered by the drone are processed separately by a simple form of machine vision software, which I have developed using the open-source Processing toolkit. This software analyses the scene and attempts to draw a line where it perceives the edge of the shoreline resides. This is, by design, an uncertain process, yielding some incongruous outcomes, but any resulting errors help to articulate the functioning of the system as a constructed technical artefact, founded on the careful tuning of contingency, as opposed to a pristine, ‘black boxed’ abstraction.

The final stage of the creative process involves the data points that constitute the detected shoreline being used to inform the selections made by a poetry generation algorithm (again developed using Processing), which aims to produce outputs that are thematically responsive to the maritime aspects of the source imagery. There are multiple aspects to this final stage, but one of the most significant for me concerns how machine detected signals are remediated so as to be rendered accessible to human observers, and whether, in an artistic context, alternative strategies of data presentation can make us more sensitive to the implications of our contemporary moment.

The planned outcome of the Waveform project, ongoing through 2017/18, is the production of multiple image sequences that articulate the various stages by which the final poems emerge, compiling these into a finished artist’s book.

As a mode of research, Waveform seeks to produce a speculative media assemblage that draws together a range of technologies, techniques, and perspectives on the functioning and purpose of digital sensory systems. In so doing, it is affording me the opportunity to gain not just a practical sense of these systems as they are developed and deployed—particularly concerning how the former affects the outcomes of the latter—but also to reflect critically on what may be described as the contemporary aesthetics of the sensory. That is, to evaluate the practical and conceptual paths through which different phenomena are established as entering into a sensory relationship—of shorelines, and pixels, and poetic excerpts—and how these relations are then presented so as to be understood and rendered meaningful by different architectures of interpretation—different collectives of humans, machines, and scientific and artistic discourses.

As a creative enterprise, the technical properties of Waveform afford it a certain resonance with number of recent debates concerning the problems posed by large-scale data gathering, drone surveillance, and the algorithmic sorting and reprocessing of digital imagery. An enquiry might thus be made into the potential of Waveform to function as an activist intervention—to demonstrate the potential of drones and digital sensing to operate in ways that subvert the commercial or military impulses with which they are most commonly associated. However, as a theorist and practitioner, I remain wary of making bold claims along these lines. As much as I find it critically and creatively provoking to deploy drones to render poetry out of waves, such endeavours, at best, constitute only a very small act of resistance against the wickedly complex forces driving the social and political ills of late modernity. While I may hope that Waveform will function as a vehicle for the raising of important questions, and maybe even a source of inspiration, I would not wish to present it as a model, nor as a substitute, for the hard, difficult task of direct political engagement.

Nevertheless, this point aside (tapping as it does into perennial debates surrounding the political efficacy of art), I would submit that Waveform does offer a certain value in prototyping an alternative way of approaching and understanding the sensory. In working against the prevailing vectors of control, the absolute, and the cynical exploitation of the abstract, and pursuing lines of experimental enquiry and imagination, Waveform stands as a testament to the possibility and potential for change, reading the signals of an emerging world not what they are, or for what they should be, but for what they might be, instead.

Interested readers can follow the progress of Waveform at the author’s website, richardacarter.com/waveform, and @RichardACarter2 on Twitter.

References

Barad, K. (2007). Posthumanist Performativity: Toward an Understanding of How Matter Comes to Matter. Signs: Journal of Women in Culture and Society, 28(3), 801-831.

Carter, R. (2017). Solid State Poetry [Online]. Available at:

http://richardacarter.com/solid-state-poetry/ [Accessed 19 July 2017].

Joyce, M. (2003). Siren Shapes: Exploratory and Constructive Hypertexts. In N. Wardrip-Fruin and N. Montford, (Eds). The New Media Reader. Cambridge, MA: MIT P, pp. 614-624.

Parikka, J. (2012). What is Media Archaeology? Cambridge: Polity Press.

Pickering, A. (1995). The Mangle of Practice: Time, Agency & Science. Chicago, IL: Chicago UP.

Salter, C. (2010). Entangled: Technology and the Transformation of Performance. Cambridge, MA: MIT P.