The New Interface

This article by author, data scientist and journalist Yudhanjaya Wijeratne is one of a series commissioned as part of MyWorld, a UKRI-funded project that explores the future of creative technology innovation by pioneering new ideas, products and processes in the West of England. We have commissioned writers, academics, creators and makers to contribute a written snapshot into how artificial intelligence is changing, enhancing and challenging creative writing and publishing practices.  

The dominant form that AI takes in public perception is the chat interface. Think of ChatGPT or Gemini or Claude; a box patiently waits for input and responds to you. 

AI has had a bumbling, confusing and hostile time in the creative world. And I think that’s because of this interface we’ve defaulted to: replacing a friend on the other side of a connection. While the chat interface is intuitive to us, shaped as we are by decades of SMS, internet messaging and so on, it’s also one of the most creatively bankrupt ways of harnessing what is essentially an artificial mind.

I think we are slowly getting past this wave, this introduction. The real utility for creative work will come from AI deeply embedded within the tools and workflows that we already use.

We are now seeing the rise of more invisible or integrated AI tools. One of the most successful examples actually comes from the field of software engineering.  Klein, Roo Code, Copilot – tools, augments, that essentially fit within the same developer experience, the same interface that programmers use to write code, but framed as an agent or pair programmer that is willing to read your codebase for you and explain something, or to add a few lines here and there, or to finish out that boilerplate section while you handle the more exciting and complex bits. ChatGPT is no replacement for a skilled software engineer, but we’re seeing widespread adoption at all levels of this plugged-in agent, a specialized thing lurking in the background ready to assist rather than replace. 

In a similar fashion, consider Adobe integrating tools into the existing Photoshop interface, or of the endless integration that Google is pushing – where the Gemini models, it seems, are always lurking in the background, ready to jump out and write that email for you or to generate imagery for your slide deck. 

While I do admit that Google is perhaps more of a jump-scare than an assistant, I think what’s good about this is that we are now moving towards task-specific competence, built in a way that it can integrate into how people already work. Let’s look at what might emerge in a few specific domains:

In writing: I expect that people would start building systems that bundle word processing with AI tools that would help authors keep track of complex plots. Helping track character arcs across long series, figuring out elements of world building from notes, building structured, searchable notes and wikis as you go along, if you will; that will be such a tremendous utility for authors. 

In game development: imagine tools that let writers and game designers create characters and let them loose with their own motivations and agendas, able to navigate a gameworld in much the same way a player can. There are interesting demos, such as NVIDIA/Inworld’s Covert Protocol; but I think it’ll hit usability when it starts becoming part and parcel of the game engines that are already popular. Similarly, I also expect tools that will simplify the process of going from concept art to 3D object; I expect AI playtesting and QA to be a thing, in much the same way that automated testing became a huge part of the software development process.

In music, I expect tools that can generate musical phrases, harmonies, which musicians can then edit, arrange, and build upon; tools that can analyze a mix and suggest or automatically apply mastering processes, or help balance individual tracks within a complex audio project; AI that can generate unique sound effects based on descriptive prompts or by combining and morphing existing sounds, offering a palette of new sonic possibilities for filmmakers and game designers.

Granted, not everything happens the way I want to. The low-hanging fruit, like the chat interface, is everywhere. In writing, this manifests as endless promises to rewrite your “content” or generate stories from a prompt. Tools exist to replace concept artists altogether, or to replace voice actors with dialogue systems, or to generate songs from whole cloth. 

A charitable reading is that the world of artificial intelligence will  – at least on some level – wake up from this nightmare hallucination of replacing human labor and move back to the much more sensible approach of augmenting what we already do. At least, that’s my belief: the interface will be the creative process itself and we can kick the stupid chat box down to where it belongs – as the simplest, most hostile facet of AI.