Can you experience a lyrical situation and a poem with your own body?

Weronika Lewandowska and Agnieszka Przybyszewska in conversation: on creating poetry in VR

“You are not supposed to call it a subject, but an avatar. There’s no reality being portrayed, no setting, but a simulation!” That is what Polish poets from the Rozdzielczość Chleba group, experimenting with new technologies, proclaimed. Imagine, then, that instead of reading a poem and recreating a lyrical situation in your imagination, all of a sudden you simply become its subject. You can hear, you can see and you can move. You can feel how your surroundings affect you, and you can see how you affect your surroundings. The “here and now” of the speaking “I” becomes your “here and now”, the literary work is no longer an “artefact” but an “event”, one in which you participate with your whole body and all your senses.

Now, don’t imagine it. Just put on your VR headset, get your hands on the controllers and immerse yourself in “Nightsss”, a VR work directed by Weronika Lewandowska and Sandra Frydrysiak. Yes, virtual reality can be a literary platform, too. Yes, you can experience poetry in VR. “Nightsss” VR (the original Polish title is “Noccc”) debuted at the Sundance Film Festival in early 2021 and has been presented at many festivals around the world (recently, it premiered in the UK) and is an extraordinary piece of work. However, it dovetails the realm of new-media activities sometimes referred to as VR or XR literature, including a long tradition of poems that are, in a way, written or re-written into VR experience (from Samantha Gorman’s “Canticle”, created using the CAVE environment, which also integrated poetic experience in VR with dance, to Andy Campbell’s “Water Cave”, in which the author applies the first-person point of view in the experience of landscape co-created with the use of typography). It is not the only romance between literature and VR with a Polish touch either (for instance, Anna Nacher co-created one of Mez Breeze’s VR collaborations). 

On the other hand, neither Weronika Lewandowska nor her poem “Nightsss”, which is (as the directors put it) the “narrative axis” of the VR experience, appeared “out of nowhere”. The work on “Nightsss” VR was preceded by years of artistic practice, creative violation of the boundaries of the poetic language and blazing trails on Polish and international spoken word stages, as well as her involvement in the creation of multimedia poetry publications and work on performances (also using VR) as part of artistic residencies and scholarships abroad.

The interview presented here is a record of conversations between The Writing Platform editor  Agnieszka Przybyszewska and Weronika Lewandowska about some aspects of creating poetic experience in VR.

“Nightsss” VR directed by Weronika Lewandowska and Sandra Frydrysiak (trailer). Check the whole “Nightsss” VR team here

Agnieszka Przybyszewska: An average fan of the art of words would not say that virtual reality and literature  have much in common. And, yet, we are here to talk about the “Nightsss” VR, which I would describe as poetry in VR, or VR poetry. Could you tell us what a “reader” who ventures into putting on goggles to experience a poem should expect?

 Weronika Lewandowska: “Nightsss” VR immerses you in an interactive and computer-generated space of images, words, sounds. It is an immersive piece of work, an experience of spatial metaphors, which come into being not only between words and meanings, but also between virtual representations of various elements written in the script, then put into motion and action in a virtual environment.

AP: All right, but most people would ask “where is the poetry here?” and “how will I find myself in all this?”

Weronika Lewandowska’s “Nightsss” as a spoken poem

WL: Everything is constructed around the axis of the “Nightsss” text, which was created sixteen years ago. It is a spoken word poem that I wrote with an international poetry slam in mind and presented many times before foreign audiences. My works created at that time took into account the spatial experience of a poem, the impressionistic quality resulting from the manner in which the body performs the text. I wanted to bring not only the words but also the performative nature of spoken word poetry into VR. Therefore, by immersing yourself in the virtual reality of “Nightsss/Noccc,” you can expect to “enter” the corporeality of the person performing the text, but also to experience the poem broken down into different elements in space, to spatialize the text through sound, visual, interactive actions. The text of “Nightsss” is about a real situation (the experience of love, intimacy, contact with nature and man that preceded the creation of the poem), which came back to me while writing the script. “Nightsss” VR also makes use of this memory, which can be seen in how the virtual world is constructed.

AP: Your explanation will probably reassure  those sceptical about VR poetry. Still, it is probably worth pointing out one more thing. Shortly after the Sundance Film Festival premiere, comments were voiced that your experience was visual poetry. The common understanding is that visual poems are made up of letters. Yet, in “Nightsss” VR, there is no typography at all. Your poem functions as a text to be listened to, the viewer is immersed in a visual soundscape. What did you want to achieve in this way?

WL: In VR, the poem functions not only as a text to be listened to, but also as an interactive audio element linked to visual change and the immersant’s activity. Through the movement of the hands, one layer of the vocal can be deformed. Thus, the person immersed in the virtual world has some extent of agency – they can deform the sound environment with a visual action linked to the movement of the body in space. They also immerse themselves in a very specific domain of impressions. Images activate certain areas of memory. So do sounds. The different impressions you get – from memory and from what you are experiencing at a given moment – mix together.

AP: Is this the very moment you meant when you and Sandra talked about “mind hacking” in one of your interviews, about creating new memories in your audience?

WL: VR experience, which engages you on various levels of perception, is able to “detach you from yourself” for a moment, from various developed ways of acting and perceiving. This is what I see as a mind-hacking experience. Charles Davis talked about the de-automatization of perception. For her (and for me too), VR changes the logic of perception and shatters habits, thanks to which we are able to change (because we allow into ourselves a different path of sensory experience). Virtual space reopens us, and this “fresh” experience gives us a new memory.

“Nightsss” by Weronika Lewandowska and plan.kton

AP: All your artistic activities seem to me to be related not only to space, but also to the de-automatization you mentioned, including the  de-automatization of reception. You offer slam poetry to the audience rather than traditional book volumes. On top of that, you work with plan.kton on audiovisual poetic performances. I’m also thinking of the video for “Nightsss”, which arranges the space with typography. It all seems like a test of which medium will allow you to best convey this spatial experience. Why did you decide that “Nightsss” VR should be a six degrees of freedom (6DoF) experience and not the more “classic” cinematic VR?

WL: You know, for me, cinematic VR is something I have already, let’s say, tested, if only by projecting visuals onto the space around me. Also, I wanted it to be an animated piece, allowing you to go beyond the framework of reality and physics. I have always been interested in abstract forms. When I worked with different VJs, I really didn’t want the body to be shown in the visuals. I didn’t want a visual reference to corporeality. Instead, I wanted to transport this corporeality, this impressionistic quality in an abstract way. For me, this is how I see poetry, too – I don’t want to speak directly. Cinematic VR was not my cup of tea. I was attracted to forms that can be made unreal, fascinated by shape shifting. I have always been interested in using very abstract means to present ambiguity and emotional states. For example, animations in which figurative forms dance together in a frame to music, synchronised or unsynchronised. This composition and dynamic, the way the objects approach each other, the way they interact with each other, the way they open way to “speaking indirectly” – this is simply closer to poetry. And the six degrees of freedom gave me exactly those possibilities (apart from the interaction space itself, of course). 

AP: What meaning, or function, does ASMR have in your work? I am asking this question, because “Nightsss” VR is often described as a piece that combines animation with ASMR. What does that involve?

WL: Usually, when someone thinks of ASMR, they think of sounds, of different textures, of the tactile sensation that sound brings. ASMR as something that characterises “Nightsss” did not come up until I got down to preparing a description for the finished VR experience.  Still, “Nightsss” VR is ASMR-like on many levels. When we talk about ASMR, we mean constructing the domain of impressions for the different senses, by getting close to the body and establishing intimacy with the body. The fact that sound can transport these tactile sensations is a feature of sensory substitution. For me, movement is also an ASMR element. I once wanted to explore what the movement of abstract forms does to us: how it engages our empathy, or what emotions and states are evoked by abstract forms dancing around us like another body. The ASMR quality of movement is created by the pace, dynamics, relations between them, tensions connected with e.g. something sliding over something else, the dynamics of changing textures. All this evokes a physical reaction of our body. That is why “Nightsss” VR is an ASMR experience for me, but this notion comes here as if post factum, only when I look at what we’ve managed to do and what means we’ve used.

AP: I also wondered about the linguistic dimension of VR “Nightsss/Noccc.” You haven’t shown your spoken word poem in translation on international stages (although there are some great translations of the text). Similarly, your VR experience is presented around the world in Polish (you can also listen to the English translation at the end). Why?

WL: This is a Polish text, while the domain of impressions (perception and how we interpret performative activities) is already beyond the linguistic barriers. Mind you, the language (and its sound) is one of the elements that appear in space and that you interpret. I’ve asked myself this question before: “what do you do with a poem when you have sound and image on stage at the same time? Would projecting a translation of the text redirect the audience’s attention, thus competing with the non-verbal, affective dimension of the poem performed by the body?” I think I actually started “doing” VR before I got to know this medium in practice. 15 years of experience collaborating with other artists, including dancers, VJs and musicians – it was learning how to create immersive experiences and work in an interdisciplinary team. Also, I learnt how to reach out into different spaces with my poem. One of those first moments was what you said about letters on paper. I also tried to construct space using typography. Later, of course, it was my body that became the medium of recording and presentation. When I couldn’t create some actions and artefacts on the computer (graphic, film, programming), I created them with my body and my actions in space.

Weronika Lewandowska with plan.kton

AP: And have you ever considered VR experiments with typography? Bringing the experience behind the spoken word scene into the VR space, in such a way that the letters are also something you can interact with?

WL: No. At the moment I am more drawn to situations in which you are immersed in something that’s in between these letters. When I was working on the script for “Nightsss”, I thought about how I would like the person experiencing it to actually write the poem themselves, so that when they immerse themselves in the VR, they feel that it’s their experience and that they are as if writing from it. I interpreted the space of my experience in such a way that “Nightsss” came into being, and now someone else immerses themselves in all these things and brings out their own potential version of this text. They find themselves in the same process and emotion that I was in. My intention was to bring people closer to this beautiful experience that I had and that was captured and stored in this poem.

AP: You told me that you keep returning to your memory, that it somehow repeated in you, too. How did you imagine the audience of your text? Did you want them to interact with this VR experience once, or did you hope for multiple “readings”?

WL: I knew there were too many elements in our experience and that you get to feel some kind of inability to read: it’s just that your perception is not able to grasp everything at once. This kind of overstimulation, perceptual chaos, was very deliberate. In general, when experiencing, we don’t break things down into elements to understand what’s happening. When you experience the VR environment of “Nightsss”, it consists of different layers. You grasp whatever you manage to grasp and in this way you construct your memory. Then, you can immerse in it once more to discover some more things. First, it is simply being that you have, just you being there. That is why I want that first entrance experience to be as strong as possible. So, referring to your question, I would like it to be experienced repeatedly. Just like when you go through a poem, you read it many times and you can have lots of different interpretations, depending on how you feel and what you direct your attention to.

AP: But you can always have a poem on paper, even a visual one. In the case of “Nightsss” VR, on the other hand, you just have to experience it again, you have no way to “take” it with you.

WL: But each successive experience of “Nightsss” means also entering into the memory of that first experience.

AP: And what do you mean when you say you appreciate the power of the first experience? Have you also been involved in designing the so-called “onboarding”?

WL: I don’t really like the word “onboarding,” I would rather call it… a form of invitation or ambient perception tuning. It is about how to properly prepare the place, how to synchronise the space that surrounds you with the space you are about to experience, so that there is no such gap. That is why some of the “Nightsss” performances were accompanied by artistic installations, whose task was to tune the senses of the audience.

AP: Why is this important?

WL: I know that people who experience VR for the first time are often intimidated by the technology and the fact that they could be observed by someone. I think it should be a space for intimate experience (just like VR is for one person). I wanted everyone who enters this space to have a sense of safety, which would translate into a relaxed and open body. You can get to another level of experiencing, if you don’t have to struggle at the beginning with the fact that someone is watching you, thinking about what is outside all the time. I wrote a manual for those who want to immerse themselves in “Nightsss” VR. I included in it information that they are not going to be photographed and they are safe, but also about what activities they can expect and how they can move in virtual reality. 

AP: You care for the comfort of your audience, you want to put them at ease. 

WL: I wanted an intimate experience. And that may not be simple love at first sight. Here, you need time to get to know the technology, time to build trust in the environment in which you are going to experience. When you start to trust the situation, then you are able to open up more to the experience of virtual reality

AP: Your first VR project was a real success. Do you have advice for authors, especially spoken word artists, who want to experiment with VR?

WL: First of all, I would recommend experiencing different things, orienting yourself to multisensory reception and space, to what the relationship between body and space is and what gets recorded and stored in your memory; this sort of working with the multisensory memory of experiencing. Once a person begins to immerse themselves in their memories, they can see that they are constructed in such a way as to create an environment. The more senses are involved, the stronger the memory is. It is worth thinking about how different our perceptions are and about when entering the intimate zone, the mental comfort zone, turns into abuse. It is very important to see the possibilities and dangers that arise from the means you use. This is thinking all the time about where the recipient is, where their sensitivity is. It is good to observe how others work, so that when it comes to directing, you know what is possible. “Nightsss” VR had, above all, a great team of sensitive creators.

AP: When you talk about the whole process of creation, you emphasise the fact that you have to have a vision, that you need to know what you want to achieve and know the ways, the means, needed to complete this task (or know how to ask for them). And to find people who will do it so that it is in line with your vision. Do I understand correctly that you don’t need to be a coder at all to write poetry in VR? That you rather need to be… a poet?

WL: No, you don’t have to be a programmer. Still, without vnLab, I would have thought that I was not the kind of person who could do something like that. There was a lot of trust there from the beginning, faith in us that we could do it and, as I mentioned, there was a well-tuned team. But first of all, you have to have a well-constructed vision, or know what kind of experience you don’t want to create. To respect the other person’s psyche.

AP: In your opinion, is such a “poetic VR experience,” as “Nightsss” VR is often referred to, “VR literature”? Is there any point in using such a category at all?

WL: “I believe that “VR thinking” has always been included in my work, because I have taken space and audience perception into account from the very beginning (but I only realised this after some time!).” VR is an experiential situation, so if your intention in making literary works is to create experience and space, then you can naturally make a literary experience in VR. It’s just that I now think that the poem “Nightsss” is not a VR piece, it was just written as a spatial experience, so it somehow settled easily in those VR techniques. It is a cool, maybe even appropriate, way of constructing texts for VR: starting from concrete moments, elements, experiences. I wonder, though, what would happen if at this point I redirected my attention to writing something for VR completely from scratch. You know what I mean, in “Nightsss” VR, we have a text that has already been there, and the question is whether I, having already had the experience of constructing this environment, with the knowledge of all these means, would be able to enter a different way of writing and create something equally strong, which could already really be called VR literature. For now, it’s just looking for means and techniques, converting one action into another, translating one medium into another.

AP: Wasn’t the difficult access to VR, the sort of hermetic nature of the medium, a kind of artistic barrier for you? VR doesn’t seem to be a popular technology among poetry and literature enthusiasts. You know, I can understand, of course, that the slam space is also a closed circle, that you don’t have a wide audience there either, but in the case of VR it often happens that even when you really want to experience something, it just doesn’t work out because of various, sometimes very basic, problems with the technology. And this is discouraging and tiring.

WL: But on the other hand, there’s something cool about it all: the fact that it is a unique situation (once you manage to experience VR), and the fact that you have the opportunity, as the author, to meet people who have never experienced VR before. This is why it is so important what you offer to them for this first contact with VR. If I were to say what VR experience is good for this, I think it would be our VR, because it just makes sure to guide the person immersed in it in a subtle way, with attention to their psyche.

AP: And it’s not too long.

WL: Yes, it’s not too long, which is why it leaves you a little unsatisfied. You can of course immerse in it a few times, if you feel the need to do so. VR is not a mass medium, but the same is true about poetry. I like working with this sense of “specialness”, the fact that there’s something magical about it, that you have to make an effort to find a VR event and get there, that you have to really want to break through your fear of the new technology. I think there will be more and more VR experiences, because people are curious; the technology will become more popular, and VR experiences will enter other spaces.

AP: I wonder if we’ll actually be talking about literature, or at least poetry, in VR in the future. Thank you, both for taking the time to have this conversation and for stimulating all the senses of my reading-loving body. The experience of your poem in VR will certainly stay long on my list of memories to return to.

The “Nightsss” VR project was realized in Visual Narrative Lab (vnLab) and financed under the Ministry of Science and Higher Education programme within the framework of the “Regional Initiative of Excellence” for the years 2019-2022, project number 023/RID/2018/19. 

The Polish version of this interview will soon be published in “Techsty” magazine.

This interview arose as a part of the research conducted within Bristol+ Bath Creative  R+D Amplified Publishing Pathfinder. Realisation of Agnieszka’s research on VR and AR as literary platform was possible thanks to the funding from the Polish National Agency for Academic Exchange in the Bekker programme (grant agreement No PPN/BEK/2019/1/00264/U/00001).