#See You at Home

Authors: Uwe Brunner, Bettina Kadja Lange & Joan Soler-Adillon.

With: Burkart Schwaighofer (Developer); Juan Carlos Duarte Regino (Sound Composition); Max Happ (Installation Production); Stefan Maier (Web Development); Macario Ortega (Contribution Web Page )

22.06.2022, Barcelona
Museugrafia “La irrupció”.
foto: Jordi Play

#See You at Home – The Domestic Spaces as Public Encounter is an interactive and immersive installation about changing the role of the private space in a time of crisis. It is part of an ongoing participatory project that reflects on our everyday domestic life between private and public spheres, and thus on our relationship with living spaces in general. The project consists of a collection of hundreds of three-dimensional documents taken between 2019-2021 in over 40 different countries during the most intense periods of self-isolation and home confinement.

Following an open call, we collected hundreds of videos of private spaces that they converted to 3D objects using the technique of photogrammetry. These imperfect objects (pointcloud images with gaps and unfinished endings) are the core content of the work’s archive, along with personal statements about the time of quarantine and the changing meaning of private spaces in the form of audio pieces and text. 

See You at Home (SYAH) expands on a previous VR work, The Smallest of Worlds (TSOW), taking both its content and the form of presentation to a very different level. As a basis for its content, it uses the same collective archive of private spaces and personal statements. But while TSOW presents it in a non-linear narrative VR space, SYAH is a physical installation that includes 3D-printed objects, plotter prints on the wall, and interactive objects with QR codes leading to 360 degree virtual spaces that visitors can view through the ‘magic window’ feature of their phone.

The installation offers and interesting back and forth between the real and the virtual, not only within the space itself, but also conceptually, as the main visual content is made out of 3D renders of real spaces using the technique of photogrammetry. Thus, the real space (e.g. someone’s living room) was scanned through a video file, then converted into a pointcloud 3D object, then 3D printed, and then visited as a virtual space using the visitor’s phone as they scan the print to enter the VR visualization. Each of these videos (see supporting material) offer a window into the content of the work. 

 

Additionally, #See You at Home includes the VR work The Smallest of Worlds, since also a VR headset will be found in the installation, thus enhancing the voyage from one world to the other. Overall, the experience of #See You at Home offers a journey within a journey that explores the private spaces as its possible forms develop and change, in a back and forth between the real and the virtual that is both conceptual and experiential.

 

Selected Exhibitions

 – ISEA2022 Barcelona – Arts Santa Mònica, The Irruption. Barcelona (Catalonia). June 9 to August 27, 2022.

Segal Film Festival. New York (USA). March 1 to 31, 2022.

Goethe-Institut Peking (China). Nov 6 to Nov. 31st, 2021.

ETH Zurich, Department of Architecture (Switzerland). Set. 23 to Oct 29, 2021.

 

Links

ISEA 2022: See You at Home.

Segal Film Festival: See You at Home

ETH Zurich: See You at Home

Goethe Institut Beijing: See You at Home (Chinese) // See You at Home (German)

Magazin: See You at Home. A lecture by Uwe Brunner, Bettina Kadja Lange and Joan Soler-Adillon

Mosaic: Entrevista amb Joan Soler-Adillon (Catalan) // (Spanish)

ISEA 2022, Barcelona. Main Exhibition at Santa Mònica

Uwe’s page: See You at Home

 

 

 

L’espace qui parle

The workshop L‘Espace qui parle (the space that talks) took place on July 3rd, 2021 at the Gâité Lyrique, in Paris. Aimed at a young audience (8 to 12 year olds), the workshop consisted in showing the creative process behind the Virtual Reality work, which was by then on display at the gallery as part of theLes Ailleurs festival.

During the workshop, the participants chose among a series of 3D objects created through fotogrammetry (point clouds), and decided how to position and scale them in the virtual space. Finally, they added the narrative layer through their own voice recordings.

A the end of the workshop, the participants would see both their own and the other participant’s creations.

The Smallest of Worlds

Authors: Uwe Brunner, Bettina Kadja Lange & Joan Soler-Adillon.

With: Burkart Schwaighofer (Developer); Juan Carlos Duarte Regino (Sound Composition); Max Happ (Installation Production); Stefan Maier (Web Development); Macario Ortega (Contribution Web Page )

The Smallest of Worlds – A Social Landscape of Collected Privacy is a Virtual Reality non-linear experimental documentary. A 20m experience in which the visitor will view a series of rooms and interact with its elements in order to visualize and hear a multi-voice, multicultural and multilingual discourse that develops throughout the experience.

Each of the rooms of TSOW is made out of a series of visual and sound pieces. These building blocks, each carefully selected and positioned in the virtual space, afford a specific experience bespoke to each of the rooms. There is not necessarily a theme, but a mood and tone that is different for each. Some are more melancholic, positive, ironic, surreal, etc where every building block carries with it a specific story; a documented case on the importance of private space and how this changed during the lockdown phases of the pandemic. Be it a point cloud image or a sound fragment, each one constitutes a unit of meaning in its own right. And at the same time, they create an overall unit of meaning at the node level.

Conceptually, the unfinished nature of the point clouds fits very well with the micronarrative approach at the basis of the storytelling strategy of the installation. The main idea is that the lack of completion – in terms of image or story – is an invitation to fill in the gaps, thus affording an experience that stimulates the imagination of the viewer. Visually this comes across through the undefined nature of many of the figures, the gaps they present and the diffusion of their limits. The parallel narrative device lies on the fragmentation of the testimonies, the disconnected nature from one statement to the next and the lack of a specific context or visual anchor for each.

However, this fragmentation is not presented in a random fashion, but rather through a careful design that includes both the construction of spaces and the narrative side of the piece. This work aims at documenting a collective experience, and thus the approach was to maintain this collective spirit also in the actual content of the work. At the core of this narrative approach is the idea that there is not just one story or character, but a multiplicity of both.

 

 

Selected Exhibitions

– Les Ailleurs. Le festival qui explore l’immersion. Official Selection. La Gâité Lyrique, Paris
(France). April 6 to July 18, 2021.

– VRHAM! Virtual Reality & Arts Festival. Official Selection. Hamburg (Germany). June
6 to 12, 2021.

– CPH:DOX, Copenhagen International Documentary Film Festival, 2021. Official Selection.
INTER:ACTIVE. April 21 to May 21, 2021. Online.

– DOK Leipzig 2020, Festival for Documentary and Animated Film. Official Selection,
DOK:Neuland official, Extended Reality: DOK Neuland. Museum of Fine Arts, Leipzig
(Germany) October 27 to November 1, 2020.

Awards

– Best Content 2021, awarded by Eurofest XR at Sterereopsia 2021, Brussels (Belgium)..

– Special Jury Mention. Les Ailleurs 2021. Paris (France).

– Honorary Mention. DOK Munich 2021. Munich (Germany).

Art residencies

– Pixel:Bytes + Film. Austria. Resident artists. 2021-2022.

– CPH:LAB 2020-21. Project development lab. September 2020 to April 2021.

– VRHAM! Online 2020 Festival. Resident project. Hamburg (Germany), June 5 to 8, 2020.

Links + Press

The Smallest of Worlds: https://thesmallestofworlds.com/

The Smallest of Worlds (Instagram): https://www.instagram.com/smallest.worlds/?hl=en

Les Ailleurs – The Smallest of Worlds: https://lesailleurs.art/en/oeuvre/the-smallest-of-worlds-2/

La Gaîté Lyrique – L’espace qui parle: https://gaite-lyrique.net/en/event/lespace-qui-parle

CPH:DOX – The Smallest of Worlds: https://cphdox.dk/lab/2021-smallest/ 

DOK Leipzig – The Smallest of Worlds: https://www.dok-leipzig.de/en/film/smallest-worlds-social-landscape-collected-privacy/archive

VRHAM – The Smallest of Worlds: https://www.vrham.de/en/the-smallest-of-worlds-2/

Uwe’s page: https://uwebrunner.com/The-Smallest-of-Worlds 

ORF: Pixel, Bytes + Film – Artist in Residence

IRL: The immersive art of Joan Soler-Adillon, artist and researcher, lands at the festival Les Ailleurs Paris

Mosaic: Espais que parlen: The Smallest of Worlds

FishEye Magazine: Éric Chahi remporte le Grand Prix du festival Les Ailleurs !

Exibart: SELPHISH. Arte digitale e società: parla Pau Waelder

Le Point: Réalité virtuelle : on a apprivoisé des méduses

El Correo Gallego: El arte inmersivo, la posibilidad de vivir una transformación

Audiovisual451: Varias producciones españolas participan en festivales de Rotterdam y Dinamarca

Fisheye Magazine, 46. Interview; Uwe Brunner, Bettina Kadja Lange and Joan Soler-Adillon. Pgs. 108-111. May 2021. (https://www.fisheyemagazine.fr/store/en/product/fisheye-magazine-46/)

 

 

Self and Other Liquids

Authors: Roc Parés and Joan Soler-Adillon

Self and Other Liquids es and image created in collaboration with Roc Parés for the panel Facing Uncertainty and Its Discontents, which was part of Ars Electronica 2020, Barcelona Garden.

The image is created from a frame of Roc Parés’ work Self&Other processed through Joan Soler-Adillon’s work Liquid Video.

The participants in the panel were Tere Badia, Marina Garcés, Roc Parés and Joan Soler-Adillon, amb Pau Alsina as moderator.

Occultation

with Uwe Brunner and Bettina Katja Lange.

Title:                   Occultation
Authors:             Uwe Brunner, Bettina Katja Lange and Joan Soler-Adillon
Format:              VR experience (Room-Scale)
Duration:            8-10 minutes

Occultation was exhibited at the Espronceda, Institute of Art & Culture gallery, in Barcelona, from February 21st to 29th, 2020. 

 

Occultation is an immersive XR installation between physical and virtual world capturing the material character of our everyday stories.

Our everyday life is told by the things we surround ourselves with: a traffic sign at the crossroads, the grandmother’s inherited camera, a newly bought Nike sports pants. Every moment in our life is much dictated by the relationship we have established to these things. We own them and we dream of them; we collect, we love, we bestow… we share our most imitate moments with things.

Private collections are witness to the existence of a person and can tell us about their story, their wishes and hopes, their preferences and maybe their fears. Things became the archives of our social life, storages of our vivid memories and canvases onto where we project ourselves.
Objects are virtual portals to the stories and histories of humankind, as much as they are a fertile ground for speculations. It is part of their character to mystify and to store the best kept secrets. It is in the nature of things that they are inexhaustible and inscrutable.

Based on a collection of everyday objects, Occultation presents an immersive experience that continuously oscillates between the physical and the virtual space. In it, visitors are able to interact with objects that expand into the virtual space and gradually unfold fragments of their untold stories and secrets. By inhabiting those spatial stories, visitors find theirselves in an intricate web of entanglement, open to speculation, of everyday objects and their owners.

Occultation was created during the Immensiva VR/AR Residency at Espronceda, Institute of Art & Culture, Barcelona (Catalonia), on Februrary 2020.

In Pieces VR

 

 

Title:                   In Pieces VR
Format:              Experimental documentary / Room-scale Virtual Reality experience
Duration:           10-12 minutes

Synopsis:

In Pieces VR is the first instance of In Pieces. Sketches of a Dystopian Present, an on-going project that aims at exploring new storytelling forms to address current political crises. This first instance is an experimental Virtual Reality-based documentary about the personal experience of being a political prisoner, or a direct relative of one, based on the case of the Catalan activists and politicians. It explores the voices of Jordi Cuixart, an activist who spent 2 years in pre-trial detention, and now has been sentenced to 9 years in prison, his partner Txell Bonet, and Anna Forn, daughter of a former cabinet minister, also imprisoned since late 2017 and sentenced to 11 years.

 

The work: 

In Pieces VR is a 10/12 minute experience that presents a concentric narrative. There is a starting space with four navigation options, each taking the visitor to one of four different scenes. After each one, the visitor will be taken back to the start, but the corresponding navigation device (a plinth that the user has to knock down) will have disappeared, thus forcing her to choose a new option. Only after the four scenes have been viewed, the visitor will be taken to a final space that shows contextual information and credits.

 

The experience creates a heavily abstracted series of spaces with wire-frame(ish) virtual sculptures that become animations and interactive elements through which the user can move around, while listening to very personal accounts of the documented case: a walk on the prison yard, a train ride to Madrid, or the awkwardness of one of the firsts family visits.

 

In Pieces VR looks at a local example of a global phenomenon: political prison and, more generally, political repression. It aims at the global audience, particularly with the approach based on abstraction and micronarrative. It is a strategy that, intentionally, leaves plenty of room to the imagination of the viewer, thus affording her to make her own connections with the personal and political context. The working hypothesis here is that this will create a very different emotional experience than that we can achieve through seeing a documentary on a specific, but after all alien, case.

 

 

This is how the portrayed characters in the documentary acquire their meaning, as a hybrid between a specific case and whatever the viewer projects to it. At the same time, In Pieces VR is an interesting example of how a VR artwork that is so far from realism can create an intense sense of place and, according to several visitors, a very powerful experience.

 

Exhibition:

  

In Pieces VR was first exhibited at Gazelli Art House in London, from October 16th to October 31st.

It has later been exhibited at the following venues:

  • Jump Into the Light VR Cinema in New York City, February 21st, 2019
  • Centre for the Study of Emotion and Law, launch event. Royal Holloway, University of London. Bedford Square, London. March 22nd, 2019.
  • House for Democracy and Human Rights. April 23rd-24th. Berlin, Germany.
  • Escola Superior de Música de Catalunya (Catalan College of Music) in Barcelona, November 29th, 2019.

 

 

It has also been featured at the following talks and presentations:

  • i-Docs Conference (March 2018; Bristol, UK).
  • New Narratives Collaborative Symposium – Ravensbourne / Royal Holloway (July 2018; London, UK)
  • Conference Keynote at Bridges of Media Education 2018 (September 2018; Novi Sad, Serbia)
  • The Immersive Kind Collective event, Hobs3D Digital Foundry (November 2018; London, UK).
  • Movement and Immobility: Catalan Political Prisoners and exiles. Who are we? Symposium. Tate Modern, London, UK. May 2019.

 

 

 

 

Amb Títol

Amb Títol is an interactive documentary directed by Neus Ballús and produced by Universitat Pompeu Fabra and Corporació Catalana de Mitjans Audiovisuals. It presents a series of testimonials of University experiences and how they can be life-changing. It aims at promoting the debate around the role of the University (and especially of public University).

[In Catalan, “títol” means both title, as in the title of a work, and also degree. Thus the name “Amb Títol” is a pun that exploits the ambiguity of having a degree and of a work that, oposite to the many ‘untitled’ ones, does have a title.]

Language: Catalan/Spanish. English subtitles available.

Role: Interaction design, project advisor.

ambtitol.cat

Transmedia: The documentary, which celebrated the 25th anniversary of Universitat Pompeu Fabra, intendedly avoided to feature any of the alumni celebrities. In dialog with this, some of these celebrities participated in a series of ‘I am angry that I am not in the documentary’ videos. See e.g. (in Catalan):


See the whole list of videos here: https://www.youtube.com/playlist?list=PLZSWqxOGTnJ1vIof0wuAiIZSwWcJYXRLX

Cadenes Humanes (Human Chains)

Autors: Laura Picallo, Justí Torn, Marcos Taboada, Núria Quero i Joan Soler-Adillon.

The interactive installation Cadenes Humanes (Human Chains) is a collaborative game were participants need to hold hands in order to connect columns distribuited in the space.


Cadenes Humanes at Festival Ingràvid, Figueres (Catalonia). 2013

It originated as a class project in the Videogame Workshop for Communication Studies in Universitat Pompeu Fabra. After a technical reelaboration it was presented in Ingràvid Festival in Figueres (Catalonia):

http://www.festivalingravid.com/programa2013/instalacions2013/

 

Cadenes humanes at the UPF campus, May 2013

The installation was made with Processing and Makey-Makey.

1st Videogame Demonstration of the UPF’s Audiovisual Communication Studies

Presentation at Universitat Pompeu Fabra’ Campus de la Comunicació of the videogames developed by the Audiovisual Communication students, using ProcessingMakey-Makey i MouTe.


Image by Laura Picallo.

 

 

Official poster by UPF.

And here is a video of a previous in-class presentation:

Press:

http://orladiari.cat/2013/05/la-upf-organitza-la-1a-jornada-de-demostracio-de-videojocs-de-comunicacio-audiovisual/

http://orladiari.cat/2013/05/a-pitchar-el-pong-una-aposta-pels-videojocs-col·laboratius/

http://orladiari.cat/2013/05/un-joc-classic-un-de-supervivencia-i-habilitat-i-un-dinteraccio-amb-el-so/

http://orladiari.cat/2013/05/real-fight-simulant-un-combat-desgrima/

Interactive Visions (LOOP 2012)

https://art.arts.uci.edu/news/simon-penny-screen-barcelona

INTERACTIVE VISIONS — Simon Penny & Joan Soler-Adillon
Does interactive video as such really exist? What is it?
At the headquarters of the LOOP Fair festival.
Pelai 28, Barcelona.
Friday, June 1, 2012 | 5:30 PMInteractive Visions aims to analyze the role of interactivity in relation to video, and, therefore, the possibilities and reasons for the existence of interactive video. While video is essentially a pre-recorded image, the more interesting interaction, as an artistic proposal, is that one that can find the right balance between predictability and randomness. This type of interaction must be defined and put in the context of the moving image that, strictly speaking, is neither video-art nor video-games. The place of interactive video is somewhere between these two disciplines that are well established and consolidated.

Tutorial: Com elaborar un treball acadèmic

‘How to create academic work’. Interactive tutorial created for the Universitat Pompeu Fabra’s Library.

The project was made in Flash, and is no longer available (http://stpr.upf.edu/tutorial/)

Role:
  • Interactive design and script.
  • Content production.
    
    

Here’s an article that explains the project with some detail:

https://arxiu-web.upf.edu/hipertextnet/numero-11/Tutorial_interactivo.html


 

 

The project was featured in this chapter of La malla tendències (minute 9):

Processing Tutorial

The processing tutorial exists in Catalan ans Spanish. It is an on-going project that started with Hangar’s workshop in 2006.

Catalan Tutorial: processing.joan.cat/cat

Spanish Tutorial: processing.joan.cat/cs

The tutorials have been used in many programs and teaching events, both by me (the first two) and others. Here’s a list of those I know of:

·Idec-Universitat Pompeu Fabra: University Master in Digital Arts
·Idec-Universitat Pompeu Fabra: Postgraduate Programme in Information Visualisation
·ELISAVA School of Design (UPF)
·INS Vilatzara de Vilassar de Mar, Catalonia
·Institut Escola Costa i Llobera, Barcelona
·IUNA Universidad Nacional de Artes Visuales, Buenos Aires, Argentina
·Instituto Politecnico San Arnoldo Janssen, Posadas, Argentina
·Universidad de Castilla-La Mancha, Spain

 

Workshop: Processing and Arduino II, at Camon Alacant

video

MasterClass in Camon: Processing, programming for artists and designers

One month after the 2009 workshop I went back to Camon for a MasterClass.

The Processing examples I used are here: http://joan.cat/processing/camon/masterClass_camon.zip

… and here is what’s left of the documentation at the Camon website:

https://web.archive.org/web/20120630125126/http://www.tucamon.es/contenido/processing-programacion-para-artistas-y-profesionales

Robot Factory

Robot Factory, an application for the Interactive Slide of the Universitat Pompeu Fabra’s Interactive Systems Lab.

I programmed both the game environment with Virtools and the computer vision system with Max/Msp/Jitter

ISOLDA

Integrated System for OnLine Distribution of Audiovisuals.

ISOLDA is a tool to edit conference events in real-time, including two video feeds, a syncronized powerpoint (or similar) presentation, original audio and the simultaneous interpretation feed.

I fully programmed botht he real-time editing interface and a postproduction interfaces in Max/Msp/Jitter, version 4.6.

scla@spciomvstr

scla@spciomvstr was a workshop on video and animation creation for High-School students, developed at Barcelona’s Espacio Movistar between may 2007 and december 2008.

During the different phases of the workshop, students produced stop clips in stop motion, photographic montage and cellphone video.

The following clip is from a youth-oriented news program from the Catalan National TV, aired in 2007 (in Catalan):

http://www.youtube.com/watch?v=Z6ympHm6kAA

 

… some pictures…

 

… and the press:


http://www.lamalla.cat/digitals/article?id=171243

The Punch and Verse

The Punch and Verse was a performance created with Ana Busto and Mark Buccheri.

“A multimedia performance combining a boxing training session and contextual imagery around Allen Ginsberg’s poem Hum Bom!. The Boxer Santana and his trainer perform a warm up session while a recording of Ginsberg’s recitation is played.” [http://www.anabusto.org/boxingPerformances.html]

Vídeo of the firs presentation fo the performance (ITP, desembre de 2004):

nou_id

Programming of a real-time 3D interactive environment for the Konic’s project nou_id.

Role: Creació of the 3D a real-time infographis (with Virtools).

Here’s some video documentation of the 3D environments that were controled in real-time during this section of the show:

Hora Fosca / Vanitati Mundi

Videoprojeccions for the Kitsch Electrokàustic show:

Hora Fosca (Dark Hour), La Pedrera Auditorium, Barcelona, 2006

Vanitati Mundi, La Pedrera Auditorium, Barcelona, 2006

And here’s the raw video for the Hora Fosca projection

Far

Performance: Real-time image generation and manipulation with Max/Msp/Jitter.

Fas was presented in several spaces during 2006 and 2007, including Barcelona’s Metrònom or Niu Space. Here’s a complete list of venues:

http://www.dtic.upf.edu/~csora/far.html

…and here a fragment fo Metrònom’s performance:

http://www.youtube.com/watch?v=geOsgVX47OA&feature=plcp

PixelDraw

PixelDraw and PixelDraw 2.0 are two versions of an online collaborative drawing tool created for the Creation Center Can Xalant in Mataró (Barcelona), to celebrate the Internet Day in 2006 and 2007.

Unfortunately, the web was hacked a while back and has been unactive since.

This is how the main page and the drawing interface for the second version:

Presó Mental

This video was produced with one camrera’s footage taken at Kitsch‘s performance in Manlleu (Barcelona), on January 8th, 2005.

Kitsch, with over 25 years on their back, is the most original and powerful Catalan band. They have presented their dark poetic lyrics along the always surprising evolution of their music. From their own interpretation of goth punk at the end of the eighties and early nineties, to more undefineable stiles and an acoustic album on the late nineties to Rock with electronic percusion at the begining of the 00s. They are now working on integrating visuals in their new acoustic show, that will be presented next fall in Catalonia, where I am going to be one of the collaborators.

Digital Babylon

Digital Babylon is an interactive installation created in 2005.

An extensive description of the Piece was published in 2011 in the book Impresión Expandida / Expanded Print, published by the Universitat de Barcelona, within the chapter Playing with Pixels. The complete reference and the descriptive text are below.

‘Digital Babylon’ is an Artificial-Life interactive installation in which the visitor’s presence alters the work’s computer generated ecosystem

 

This piece gives two choices to the visitors: contemplation and interaction. It is a digital ecosystem that works on its own without the need of human intervention. There are two creatures (the triangle beings and the roundheaded tailed predator) and a plant, that interact with each other, eat, kill, die, mate and reproduce. By doing the latter, the two animal’s species evolve.

If the visitor decides to intervene, the action will be done with his/her body, moving around a designated space. When doing so, the visitor’s body’s position will be mapped on the virtual world, and the triangle beings will be more or less attracted to the visitor, depending on how much friendly each one of them is. At this point, the participant decides either to help or to harm the triangle beings. Keeping them away from the predator will be helping them, and dragging them to the predator (to death) will kill some of them and harm the species. Depending on this actions, the next generations (where friendliness is one of many factors that the virtual parents pass on to their children) will be affected, becoming the species as a whole more or less friendly with visitors and, therefore, more or less inclined to interact with the human beings visiting the space.

 

 

Some documentation videos:

The ecosystem:

Visitors in the installation:

A simulated visitor:

Evolution: An example of the change for behavior (through evolutionary programming) after a 13 hour run:

Detail: An individual of the main species eating, dying:

Detail: Reproduction in the main species:

Detail: Predators go back to the nest:

Detail: Predators attack:

Digital Babylon is an interactive installation that is very different to the previous ones. In the first place, its aesthetic, which abandons work with real images in order to concentrate on synthesised ones. Synthetic and simplistic images, as the elements that make them up are intentionally simple so that movements and interactions can give the piece its intended feeling.

The piece presents the viewer with a small virtual ecosystem. The system works without the need for the viewer’s intervention, although she can actually intervene in order to alter it. by entering a designated space in front of the projection that presents a virtual space. This installation was born out of the will to propose an alternative to a process that was commonly found in interactive installations. That is, one of a clear duality of states: a first state of waiting, always identical, and a second of interaction. Usually, once the second state is over, the piece simply returns to the first state. Digital Babylon proposed to try and make an installation that would accumulate each and every one of the audience’s interactions, to offer a proposal to the viewer that is always changing. The point was to offer the viewer two levels of interaction. A level where one finds and understands immediately, and another, more subtle one, closer to the idea of a piece that converses with its surroundings, that we might call cumulative interaction[1].

The work, inspired by works such as A-Volve by Christa Sommerer and Laurent Mignonneau, or Karl Sims’ Evolved Virtual Creatures, offered a double level of interactivity. One is obvious, with the viewer entering the installation space, and another subtler one of accumulation, developed through evolutionary programming. What follows is a description of the work and its processes.

Upon entering the installation space, the user finds himself before a projection of a black background. Within it, two types of visual elements are moving and a third one gradually appears. They constitute the two species that live within this virtual ecosystem with a third element that they feed on.

 


Digital Babylon’s Virtual Ecosystem

The first species (which we shall call main species) is made up of between a few and a few hundred of simple looking beings (small triangles that change colour when their state changes). Its basic functions are to feed and breed. If they don’t do the first of these enough, they die. And if they do it up to a certain level, they can accumulate enough energy to attempt the second function. If the attempt proves to be fruitful, a new being of the same species is born and enters the chain. All of this is done whilst avoiding contact with the other species, which is its predator, as much as possible. The second species also has its own different ways of feeding and breeding. It feeds at the expense of the main species, as it is by eating them that it obtains its food.

The main species finds nourishment in plants, which appear where the lifeless body of one of the members from either species lies, as if its body were a fertiliser for the digital land. The predatory species is aware of this, and always moves around the areas populated by plants. If they find a member of the other species on their way, they will attack it to trap it and eat it.

All these rules and interactions between each individual element generate a considerably complex system, which in its own way creates its own balance and rhythm. There are phases where there is a lack of food, which lead to the death of many individuals, which in turn generates a large number of plants, and, consequently, a new phase of abundance.

Within this whole process, one may observe traces of the afore-mentioned concept of emergence, as complexity sometimes appears to go beyond the explanation that comes from the mere analysis of the parts that make up the work, especially if we consider one last aspect that is yet to be explained.

Both species evolve through genetic algorithms. a computer technique that was first described by John Holland in 1975 and then used by artists linked to A-Life Art (art made by generating artificial life systems). These works applied basic ideas of genetic evolution to programming. In short: genetic algorithms usually begin with a population of elements (agents, programming subroutines, etc.) made in an arbitrary way within a number of predefined parameters (the genotype): each of these elements (phenotypes) is then evaluated according to predetermined criteria (fitness); the most successful elements are selected to then create a new generation of the population, which will be evaluated again, and so on. The variables that make up each of the elements act as virtual DNA, which is re-combined in consecutive generations. In this way, new individuals, even if they are not the same as their progenitors, inherit the characteristics that have made them successful. Finally, there is also the possibility of applying mutations. That is to say, to randomly alter one of the variables that conform some of the individuals. This allows for the insertion of new possibilities into the system that are different to predetermined ones, and which, if efficient within the environment they emerge in, will enter the evolutionary process.

All of this is applied to the two species of Digital Babylon. Every time two individuals of the main species mate, a new individual appears, inheriting their abilities. Here, as in other A-Life Art pieces, there isn’t a predetermined fitness criteria, Their success or failure depends on their interactions with the rest of the elements within the virtual ecosystem and, as we shall see, on the viewers’ actions.

With regards to the predatory species, the process is very similar. Although the reproduction process depends on its actions as a group, the result is the same: only individuals who obtain a certain amount of success within the environment will be able to reproduce.

All of this makes up a complex and, at the same time, changing environment. Both species are constantly evolving. When only a certain type of individuals reproduce, the species as a whole gradually changes, and this in turn affects the rival species [2]. If we finally add the audience’s actions to all this, we may come to an understanding of the idea of a double level of interaction. The way this interaction is produced is, simply, through the user’s presence within the installation.

There is no other interface apart from the body of the viewer. His presence within a determined space in front of the screen leads to the appearance of a small dot that represents him within the virtual ecosystem. When this happens, and as long as they are not too busy eating and trying to mate, each of the individuals of the main species will tend to come more or less close to the newcomer, depending on their virtual DNA code. So, some individuals will completely ignore him. and others will follow him everywhere. From this moment onwards, the viewer can decide how to interact with the individuals that follow him. He may decide to help them, or might do the opposite, which means he will put his followers in danger of being eaten by the predators.


A Visitor in Digital Babylo

Depending on what she does, she will make the individuals who are more prone to come close to visitors, or the more unsociable ones, have more or less possibilities of survival (and, therefore, of reproduction and of passing on their characteristics to consecutive generations). He will affect the interactions of future users of the work, making it easier or more difficult to interact with the individuals of this first species, depending on the sum of all the users’ interactions.

It is in this sense that we can talk about a double level of interactivity: first, there is immediate interaction, where elements react to the visitor’s presence; secondly, there is cumulative interaction, which is made up by each and every one of the users, and which makes the piece change a little with everyone, without any of the subsequent interactions having the capacity to eliminate the effects of those which were produced before it.

This double interaction, together with constant change due to genetic algorithms, is Digital Babylon’s main proposition. The point is for the work to modify each of the individuals’ movements; how members of the same species interact, how they interact with those from other species, with the environment, and how those of the main species interact with the user of the piece, as a result of all the different interactions and of evolutionary programming. The intention is that in this way, if the installation is active during a long period of time (various days or weeks), the changes might be substantial enough, for a viewer who might return to perceive the work differently, and find it interesting once again.

 

——-

Notes:

1. The notion of cumulative interaction is described in more detail in the first article mentioned in Soler-Adillon, Joan (2010). “Emergence and interactivity: A-Life Art as a paradigm for the creation of experiences in interactive communication”. Hipertext.net, 8, http://www.upf.edu/hipertextnet/en/numero-8/a-life_art.html. .

2. The results of these processes can be very surprising. During a particular stage of the process, the Digital Babylon prototype was left working for three hours in a row. After this time had elapsed, whilst in a group, the main species moved and (apparently) acted in a way that was surprising, even for the actual programmer.

Soler-Adillon, Joan (2011), “Playing with Pixels”, in Impresión Expandida / Expanded Print, eds. Eloi Puig, Alicia Vela and Antonia Vilà (Barcelona, Universitat de Barcelona, Ministerio de Ciencia e Innovación and European Regional Development Fund, ISBN: 978-84-614-8786-8), 169- 176.
Translation by Alex Reynolds.

Painted

Painted is an interactive instllation created in 2004.

An extensive description of the Piece was published in 2011 in the book Impresión Expandida / Expanded Print, published by the Universitat de Barcelona, within the chapter Playing with Pixels. The complete reference and the descriptive text are below.

Painted was made after Liquid Video, and, in some way, can be understood as an evolution of the latter towards something that is actually an interactive installation. Painted was strongly inspired by works such as Daniel Shiffman’s Swarm and Rozin’s mirrors.

Just like in the previous piece, this work is a digital painting thattries to converse with its environment, but in this case it also starts up a dialogue with the viewers; in fact, Painted only converses with its environment when it detects movement – a symbol of life, or at least of activity – within it. Similarly to the way a work of art does not exist without eyes to look at it, this work only exists with the presence and movement of the viewer’s body. If nothing happens in front of it, nothing within Painted changes. If something moves, a section of the image will change so that it becomes impossible to come close to the work without altering it. It is therefore a work that constantly looks for change in order to respond to it. This is the dialogue it offers. If nothing happens and nobody moves, the piece also remains inactive.

This work, just like Liquid Video, uses pixels as its raw material and video as a source of information. It also never updates a frame completely. In Painted, only a small section is updated (changed) at a time, based on a specific point. In fact, it uses the point, or one of the points, where movement is detected, so that the viewer is also subject to the rhythm of the piece.

Each section is repainted radially, from the interior to the exterior, as if it were a centrifugal explosion of digital paint. Furthermore – just like in Liquid Video -, in the redrawn area, not all the pixels are updated, some are, some are not (once again, randomness is used). This gives the painting a sort of pointillist appearance that makes it more interesting.

 


Painted

The fact that only a single point can change the image at any given moment also allows the user to only leave behind mere traces of his presence. The video that is at the foundation of it all therefore remains in the background after creating an almost painterly image. Consequently, this is a piece that continuously looks for change and movement, and only acts when it detects it. Just like Liquid Video, Painted is a piece that demands the same space of a conventional painting, but also offers interaction. Its ideal situation could be an art gallery, but also the lobby of a busy space or even a street, as I already described above.

The piece has an aesthetic value, as its presentation offers a sort of image that is much closer to the art of painting than that of television. It is simply an invitation to look. But it also invites participation and dialogue. If the viewer so desires, he can, on his own or in a group, play with the making of the image. The rules are simple enough to learn, or to sense, quickly, and the result is elaborate enough for time to extend through an act of exploration, which is in fact a participation in the very creation of the piece.

Soler-Adillon, Joan (2011), “Playing with Pixels”, in Impresión Expandida / Expanded Print, eds. Eloi Puig, Alicia Vela and Antonia Vilà (Barcelona, Universitat de Barcelona, Ministerio de Ciencia e Innovación and European Regional Development Fund, ISBN: 978-84-614-8786-8), 169- 176.
Translation by Alex Reynolds.

On Beauty

On Beauty is an interactive instllation created in 2004.

An extensive description of the Piece was published in 2011 in the book Impresión Expandida / Expanded Print, published by the Universitat de Barcelona, within the chapter Playing with Pixels. The complete reference and the descriptive text are below.

On Beauty is the third and last piece of this first series. It is a proposal that asks for the viewer’s participation in a more direct way, and, in this sense, is a more conventional interactive installation. Following on from previous works, On Beauty presents a space with a white screen to the viewer, which is surrounded by the subtle projection of an image that is being processed from a life feed of a camera placed behind the user. Next to the screen, the viewer will find a paintbrush with a small built-in light. By moving this paintbrush towards the screen, he will see how a still image unveils, following his movements. This image is worked on through algorithms that are similar to those used in Painted to present a real image in an almost painterly in a sense, embellished, style. In this way, the viewer finds himself in a technologically advanced environment (a plasma screen. a projection. a modified paintbrush …) and some images that are, at first sight, just pretty images.

The surprise lies in the content of the images, which is revealed by moving the paintbrush. These contain images of children and adults injured at war, and of victims of police abuse. Images that represent social injustice in places that are far from advanced societies, where the heavy presence of technology is often sustained by different conditions in other parts of the world.

 

 


On Beauty

Hence the name of the piece: On Beauty is a modest claim, a small outcry against the unquestioned acceptance of technology’s pre-eminence in advanced society, which often goes hand in hand with opulence; the unnecessary and unbridled consumption of products that have been fabricated under conditions that we would find, in theory, unacceptable.

In fact, there was a clear element of provocation within the work, and the question of whether it might be too risky a piece, whether it might anger some viewers upon realising that they were painting injured children, was raised. The fact is that throughout the two days of the initial presentation, not a single user appeared to bat an eyelid. Many didn’t even realise what the image they were revealing actually represented, perhaps because they were not looking for a figurative image within it. Perhaps, also, we are already immune to this sort of images.

On Beauty concluded this small trilogy, which was based on the handling of real images. This third piece, which was different to the first two works in terms of presentation, also opened the door to proposing a greater implication from the viewer. All of this influenced the fourth and final piece presented in this text.

Soler-Adillon, Joan (2011), “Playing with Pixels”, in Impresión Expandida / Expanded Print, eds. Eloi Puig, Alicia Vela and Antonia Vilà (Barcelona, Universitat de Barcelona, Ministerio de Ciencia e Innovación and European Regional Development Fund, ISBN: 978-84-614-8786-8), 169- 176.
Translation by Alex Reynolds.

Liquid Video

Liquid Vídeo is an instllation created in 2004.

An extensive description of the Piece was published in 2011 in the book Impresión Expandida / Expanded Print, published by the Universitat de Barcelona, within the chapter Playing with Pixels. The complete reference and the descriptive text are below.

Liquid Video is a piece that came about as a result of working with pixels in the way I described above. It is, at the same time, a modest example of something that is also related to the last piece that I will present within this text: the concept of emergence, which describes complex phenomena generated by repeating simple rules and processes [1].

In fact, Liquid Video isn’t really an interactive work. The term ‘reactive’ doesn’t define it correctly either. The most appropriate term to describe it would be that of a live painting, or a mirror. It is a work that converses with its environment [2] regardless of whether the environment contains a human presence or not. In this way, the viewer can choose whether to participate in the constant creation of the image that develops within it or not. If he does, he has to adapt to the rules and rhythm of the piece, as its process will continue regardless of the image being read by the camera.

The installation works as a continuous loop; a coming and going between two states that can be defined, from their appearance, as liquid and drying states. The piece consists of a projection screen and a video camera that feeds the system with the images it works with. In this way, always using images from the video, the first state presents the visitor with a reflection that is increasingly distorted. The colours smear along the digital canvas as if a ‘water effect’ had been applied to them until shapes are unrecognisable, and the only recognisable features that remain are the actual colours of the environment and the viewer.


Liquid Video: ‘Water Effect’

At this point, the system enters the drying state, and the image from the video gradually pulls itself together again. The colours slowly return to their original place, but leave small traces and strokes behind them along the way. In this way, the resulting effect is similar to that of a painting drying, which is created by using the image of the mirror that emulates the video. Just when the image begins to look like a simple replica of what we would see through a conventional video, the process of the water begins again.


Liquid Video: Drying stat

The process that creates these two effects is actually extremely simple. The system is only reading a limited number of pixels that make up the image of the original video (just 10,000 out of a total of 307,200), but instead of painting them where it should, it places them on a point that is generated by a basic programming concept: the Random Walker.

The Random Walker is nothing other than an element whose position is constantly moving up or down, or right or left, randomly. It is a typical exercise in introductory programming courses. This is therefore the foundation of the piece: to read pixels in one place and paint them in another. What is meant by reading and painting is the reading of colour information of a point in the video, and the drawing of this very colour on another point, keeping in mind only two rules: it should not be painted in a completely arbitrary place; a random walk would be used starting from the point where the original colour was read, and that not all the pixels should be painted, only a small amount. The only change from one state to the other lies in a slight tendency of the random walk to take the points back to their original position, where they constantly read their colour value. The fact that not all of the pixels change allows for the drying effect to emerge, which is nothing other than the traces of colour that a pixel leaves behind when it returns to its original position.

So, they are simple rules that generate far more complex results than might be expected from the mere analysis of rules, at least from an aesthetic point of view. Which is why the concept of emergence was mentioned earlier, as that is how the drying state came about. It wasn’t a planned or sought after effect, it came about when the highly simple rules that underlie the piece’s programming were carried out, with small variations from the liquid state.

The result is a painting in a state of continuous movement that imposes its rhythm on the viewer, or, rather, its surroundings. When placed before an urban landscape, it turns into an interpretation of it that reminds us of how painterly what is occurring before the camera actually is. The installation was up for a few months in Manhattan, pointing at the crossroads between Broadway and Waverly Place. By night, the taxis’ headlights would leave white and red traces on yellow. The passersby, mostly without knowing, also added some colour to the composition, and the newspaper stands would appear and fade away continuously.

Within other environments, Liquid Video can be an intimate mirror that plays with one’s own image, slowly drawing out and blurring one’s features. Leaving traces behind and evoking multiple metaphors about life and the passing of time.

 

———

Notes:

1. For a general explanation of the concept of emergence applied to the digital arts, and to consult a list of references, see Citation: Soler-Adillon, Joan (2010). “Emergence and interactivity: A-Life Art as a paradigm for the creation of experiences in interactive communication”. Hipertext.net, 8, http://www.upf.edu/hipertextnet/en/numero-8/a-life_art.html. See also Soler-Adillon, Joan (2011). “Creating Black Boxes: Emergence in Interactive Art”. ISEA 2011 Proceedings: http://isea2011.sabanciuniv.edu/paper/creating-black-boxes-emergence-interactive-art

2. The artist Jim Campbell proposed the idea of a dialogue between interactive work and its environment in an article from the year 2.000: Campbell, J. (2000). Delusions of Dialogue: Control and Choice in Interactive art. Leonardo, 33, 133—136.

Soler-Adillon, Joan (2011), “Playing with Pixels”, in Impresión Expandida / Expanded Print, eds. Eloi Puig, Alicia Vela and Antonia Vilà (Barcelona, Universitat de Barcelona, Ministerio de Ciencia e Innovación and European Regional Development Fund, ISBN: 978-84-614-8786-8), 169- 176.
Translation by Alex Reynolds.

Other Projects

Other projects. Except the first one, they all belong to my period as a student in New York University’s Interactive Telecommunications Program (Tisch School of the Arts).

Website for the book El Último Soldado del POUM:
http://adillon.com

The Plague:
http://joan.cat/media/plague/

Project 82:
http://stage.itp.nyu.edu/~js1800/projects/p82/index.html

Fooschat, with Jeremy Newton:
http://jeremynewton.com/fooschat/

Idrathernot:
http://s92730359.onlinehome.us/idrathernot/home.pl

Plante of Sound:
http://stage.itp.nyu.edu/~js1800/projects/planet/index.html

Sonia Gets Angry:
http://stage.itp.nyu.edu/~js1800/pcomp/midterm/index.html

Dialog:
http://stage.itp.nyu.edu/~js1800/projects/dialog/index.html

Presence:
http://stage.itp.nyu.edu/~js1800/projects/presence/index.html

Clay Dudes in Heaven:
http://stage.itp.nyu.edu/~js1800/summer04/nm/week3/

Matt Yule, live at ITP:
http://joan.cat/media/matt/

Portraits I:
http://joan.cat/media/portraits/

Digital Drawings (homage to):
http://joan.cat/media/digitaldrawings/index1.html