Playing the Bitter-Sweet Symphony of Technology: A Review of ON || OFF || ON: The Shop as Gadget-Heaven

The leaflet of Marloes van Son and Alex van Giersbergen’s (ALOES) exhibition titled ON || OFF || ON: the shop as gadget-heaven reads: “In the exhibition, ALOES presents a series of devices that question the idea of personal electronics, gadgets, and their mediated interactions. When a design creates expectations, humans are trained to recognize elements and predict the result that comes from using them. A range of custom-designed objects tries to expose the layer of communication that gets lost in translation when interacting through devices. How does the imagined ‘personality’ of a device influence our willingness to communicate with/through it?”

I lift my eyes from the paper and I notice a banner stating: Please Do Touch the Artworks. Clearly, the artists have designed an exhibition that wants to engage the visitor directly and hands-on, by means of interactive objects, interfaces, and tangible controllers.

I am welcomed by sounds coming from objects; short utterances, while a pulse and a groove, as a heart-beating echo coming from the second room. The gallery is inhabited by machine presence.

The first series of pieces I encounter is titled Phones, 5 wall-mounted panels with LED lights and buttons. Inspired by the development in the interface design of telecommunication devices across the last century, they look to me like clocks, blinking counters, waiting for me to be initiated or stopped. I dial a number, a code that will unlock the combination to the secret meaning of this apparatus. The design is minimal, elegant, and transparent in the way it shows the circuitry that drives the algorithm which composes the pattern of blinking light; an appearance that I am familiar with thinking about the previous work by Marloes Van Son.

The term “affordances” is very dear to designers. According to J. J. Gibson, it can be understood intuitively as the quality or property of an object or environment that defines its possible uses or makes clear how it can or should be used¹. In its original theory, Gibson does not explicitly address the action upon an object as the main concept of affordance, but rather a relational aspect between an animal and the environment where it lives; an ecological view in which the animal and the environment mutually shape each other and where the properties of the environment exist only in relation to the intention and capability of the situated observer. Only later on in the context of Human-Computer Interaction, Donald Norman appropriated the term affordances², explicitly referring to the action potential elicited by an object (which becomes a tool or a device) in front of which we stand. It is the “graspability” of a spoon, the “climbability” of stairs, the “pullability” of a rope.

In this exhibition, the artists’ trickery is to create false or misleading affordances, with buttons that want to be pushed and knobs screaming to be turned until they realise the outcome does not quite match the expectations. It is a fascinating strategy that pleases us at first because of how strongly and primally a person wishes to engage in the interaction and is subtly critical when we realise our unquestioned familiarity with technology.

The second work I encounter is Voice Assistant S.A.G., a square-shaped device with LED lights and audio. S, A, and G are the initials of our dear voice-assistants Siri, Alexa, and Hey Google. Each module plays a series of voice samples in random order. The devices seem to have more agency on their own, displaying more varied and complex patterns. I fail to figure out what knobs and push buttons are causing. The voice calls me back when I stop interacting with the objects; they demand my attention. The artists point out our attachment to interfaces that are supposed to assist our daily routines and support our amusement. Nonetheless, they conceal a level of surveillance over our activities and the environment we inhabit. Whether the lack of understanding of my interaction with the objects is deliberate and part of the concept of the piece is unclear to me. I was left wondering why I am still trying to figure out my relationship with these objects: why am I still giving time and attention to them without knowing how my actions will be used? The piece is more about hinting at the problem of unregulated user data collection than tackling it directly by opening up the process and exposing the algorithms and business model behind it.

Routes Alphabet is the next interface in the collection of objects in the shop the artists have opened. At this moment of my visit, I feel more like I am in an arcade of the 80s. A nostalgic retro vibe feels pervasive. But that would be anachronistic since this piece uses video projections of digitally created animation connected to a midi controller. The source data for the shapes displayed are the different routes of daily commutes during 2021. For the artists, “the work explores how small changes to something mundane, like the walk to a studio, can create entirely new experiences”. The projection seems like a pop art fluid simulation I can somehow influence or disturb. The interactivity of the piece utilizing knobs and switches can be appreciated after some tweaking but seems unnecessary in an exhibition context. I don’t feel attached to the pattern I am observing. The interaction does not reveal any particular property of the image I am looking at, and conceptually, there is no relationship between me as an agent in the space and the alteration of what is projected. I probably need to find elsewhere the motivation for the playability of the piece, and I find it in the fact that Marloes and Alex use this interface during the audiovisual performance as a duo. They are sharing with me the instrument they have built to go on stage.

Before proceeding to the second room, I spot coloured glass tiles as a low-res display on the floor. It is Rare Earth Elements/Metals, three wood and plexiglass poster stands retro lit using custom software. Inspired by images of a lithium mine in Chile, where pools of vibrant colours cover the landscape as pixelated images, they are data visualisation displays of the location of different mines located around the world, “a reference to a beautiful image of a destructive process”. The stylized design, orderly arranged, clashes with the provenience of the image. This is the only piece that questions the materiality of the technology we use, the ecological impact of the excavation and exploitation of natural resources, and the human labour necessary to sustain the development and production of technological commodities in our pockets. As Parikka puts it: “How is the earth being mobilised as part of the creation of media technologies, in terms of the materials and the labour involved?”³. We just get a taste of this “medianatures”, the double-bind between media and the geophysical. The piece is a reminder, almost a link to the issue, rather than diving into it. It is watching the phenomenon from afar thanks to visualisation embedded in an artwork.

I enter the second room, and I realise the groove I was hearing can be controlled using a synthesizer. It is ON || OFF || ON, an online and offline sequencer installation with code-poem and a custom website. There are eight modules on the wall. Each module represents an audio sample. A large knob on each module can be turned to change the pitch of the corresponding sample. The modules are connected to a central console, on which a small display shows the same symbols as the modules. The order in which the samples are played can be rearranged with the knobs on the screen. A slider controls the overall speed of the sequence being played. On the website, the visitor can also influence the order of the symbols and switch samples on and off. The artists want to create “an invisible collaboration between the visitors of the gallery and the website.”

You can leave the sound space in the state you want to have a strong influence on the next visitor… are the artists just playing with me? Technology is completely opaque, despite the illusion of control. The computer mirrors what I am doing, introducing another level of complexity and mediation. I find it somehow more familiar, despite the fact that it is much less tangible. The samples are human voices, which I can pitch up and down by turning the knobs of a console.

Being the only one in the room lets me freely experiment with combinations of sounds and patterns until I feel most possibilities are exhausted, the variety of samples and their arrangement in time fits the space, and the sounds are not too annoying for the Myymälä 2 staff to hear them over and over. I can create my own meditative space.

I could frame my experience as playful and puzzling at the same time, with clear intentions towards both. My encounters with these alluring interfaces have made me reconsider our relationship with devices, no matter how smart or dull they may be. Yet, and probably due to my profession, themes and realisations feel quite familiar, and I am much more drawn to wondering about the humans behind those concepts and realizations. And the material conditions that supported or endured the birth of the exhibition?

By interviewing Marloes and Alex, I got to know that the starting point came from human-machine interactions, different modes of communication, and mediated experiences. During the process of creating the works, they fine-tuned their ideas and added the context of a showroom/shop as a framework to present their gadget-like artworks.

The duo is tackling the themes of communication and miscommunication with devices, interactivity, and playfulness, and it seems to me the exhibition is best experienced together with others. The social dimension of the experience is something they work with deliberately in their practice. “The ‘ON || OFF || ON’ installation is specially designed to be interacted with by several people at the same time. By creating a layout of the control panels scattered around the room that gives a single person only limited control, we wanted to actively encourage people (that may or may not know each other) to work together to create an interesting sound sequence. The online component, which uses a shared sequencer among all the visitors of the website, is also geared towards people influencing each other’s result.”

As a whole, I can’t really position those “gadgets”, as the artists refer to the designed artefacts, waving between simulacra of personal technology or objects of desire, or, more simply put, slot machines for art-goers. They are works “that can be understood as artsy gadgets or refer to personal electronics. We presented the exhibition as a parody of high-end gadget showrooms, where we asked visitors to interact with the presented objects while questioning their everyday engagement with technology. One factor that very much influences someone’s attitude towards a device is whether they can afford it or not. Within an exhibition, this creates an interesting conflict, as many people don’t enter a gallery to acquire an artwork, they enter to experience it.”⁴

They created a space where visitors felt free to experiment, play around, and explore, where there was no “correct” way of approaching the work. In fact, “we hope visitors find ways to engage with it that we never imagined.” They intend to question the role of personal electronics in our daily lives and personal interactions. “A large amount of communication is influenced by the technological device we use to establish contact. What would happen if certain devices function counter-intuitively, or if more playful elements are added into its interface design, or if the device doesn’t communicate?”.

I think I know what is happening here: the reason behind it, the process and means of realization, and the artistic decisions made. And yet, or better said, “for this”, I am teleported to thoughts about the “work”: the very superposition of effort, material condition, time, and investment put into the realisation of this exhibition. It is TICK.

I am perplexed about the decision to make objects to show how our relationship with other and personal objects is idiotic. In other words, do we still need to create things to point out how we should not make things? Probably yes.

As artists working with technology, we can explore the most poetic horizons of this endeavour, but we should not shy away from scrutinising it from within. Despite not delving full-on into their inquiry, I feel the ALOES are on a swing, floating between those postures. And I like their dancing. Media art, whether archaeological or “new media”, posits a reflection on the impact of technology on an individual and societal level. It acts as a mirror, partially damaged or shattered, where the mediation of technology needs to become opaque as opposed to interfaces that claim to be transparent and tunnel us through the functionality directly. In the moment of brokenness, the interface exposes itself, manifests its presence, and can be looked at for what it is. Breaking, hacking, appropriating, and even remaking are artistic gestures willing to creatively expose the problematic relationship between devices, technological infrastructures, and centralized platforms.