Art & Science

I find that creating multimedia and interactive applications both provide a great combination of technical challenges and creative expression.
Before entering engineering school, I took 3 years of undergrad studies in classical arts. This was a great learning period, that provided the background for the creation of digital/tangible artifacts. I find that supporting artists as enriching, in the day-to-day making of tools for them, and with my brother Mateo Amaral as a support for his audiovisual production.
I find the experimental hands-on approach is what suits me best, as I learn during my time in research (PhD, post-doc 1, post-doc 2) and with friends and colleagues in the creation of game-jam projects.

Live Audiovisual Performances

Over the past 10 years, I have been working in collaboration with Mateo Amaral to support transmedia audiovisual performances, taking advantage of the power game engines for live interactive rendering, by allowing a virtual world to be controlled and remixed using audio control (OSC). This was used in multiple live performances and video installations, both showcased internationally. 

The project started as an extension of Mateo’s approach to VJ-ing (live mixing pre-rendered scenes, a combination of scenes from his movie project Piedra Negra, data-moshing, and abstract images). Our goal was to replace render and mix with real-time rendering using Unity (video game engine).

Live shows (2019), including the MUTEK festival and the Argentinian modern art museum (MAMBA)

The real-time rendering quickly turned into procedural generation, and then the creation of dynamic objects and simple AI entities. The current instance of this is Éramos la humanidad (“we were humanity”), a series of trans-media episodes that tell the story of a future where digital entities (AIs) inhabit abandoned servers in the river deltas of El Tigre (Argentina), relearning about their human ancestry.
The first episode of Éramos la humanidad was accepted and showcased at the Proa Fundation (Argentina). The second episode included the addition of AI creatures, first with an interactive space for the RealMix 2020 VR Festival (Colombia). This game-like experience transformed into Tu’hu, taking shape of a generative installation at the façade of an indie gallery, followed by an audiovisual live set. In 2022, Tu’hu was extended into a 3-month exhibit at the Andreani Foundation, and additional follow-up presentations are currently in the works for 2023.

Audiovisual performances

Here is a retrospective of the past 10 years, including the main projects and locations.

The framework we use is constantly evolving, mainly guided by the needs of our next presentation. We are progressively including more ML-generated content (TTS voices, textures, and maybe texts) and we would like to support direct interoperation (during the preparation, but ideally live).
Perhaps one day this custom tool will be available publicly, but certainly not in the near future.

The structure of our framework.

At a personal level, it was Mateo who first exposed me to modern technologies (including mixed reality) and artistic installations, and it has been a pleasure to be able to apply my technical knowledge to assist him.

Art and Science in Research

To a lesser extent, I was also involved in art & science projects from my research institutes. During my studies, I was involved in the collaboration between PLADEMA and Patricio Gonzalez in his project Efecto mariposa; no great contribution came from my part, but it was a direct inspiration for Inner Garden, and I will be forever grateful for being part of that collaboration.

In research, a series of experience-driven projects started with Inner Garden (an augmented sandbox containing a world in miniature connected to our physiology). After that, we worked on multiple projects in close collaboration with Jérémy Frey and his startup Ullo, and multiple of these projects (including modular physiology, soon to be published) have turned into products and showcased at CES las Vegas (2019-2020).

Inria-Potioc was also involved in multiple Art and Science projects, one of the most relevant ones is the collaboration with the artist Cécile Léna for the creation of tangible and augmented reality applications. My role was to provide advice and technical guidance to the students attached to these projects, notably Clémentine Petit.  Another axis of interest is the use of technology for didactical purposes. I assisted in the formation and the development of the application to this end, particularly with Philippe Giraudeau during his PhD. that turned into his startup Co-Idea. This was also explored in collaboration with Ullo, taking a more playful approach.