A combination of web-gl with web-audio technologies resulted in a very engaging music interaction experience
The following video is present for documentation purposes, and I encourage to visit the actual online toy before seeing the video. Play with it
Context and motivation
I started to understand that current music making is slowly merging with music listening, as the boundaries of both activities are being blurred by new interaction technologies and new music distribution technologies. I have also observed a trend where music making tools take stakes more and more towards having a unique, distinctive sound. This trend is odd in the sense that such tools are less versatile; but it makes a lot of sense when we thing of this tools as active music listening tools. This experience is very specific. It consists of a simple musical state machine, with pre-recorded tracks that are looped in sync; in a similar way to how Ableton works or also the Korg Electribes when configured in a certain way.
This project emerged organically from my experimentation with two web technologies in parallel: Web-audio, and Web-gl (thanks to Three.js). On one side, I was experimenting with different performative possibilities that web browsers offer (for example, Ms Compose 95), and in the other side I was working with Three.js for some client-related work (e.g. Tzina: Symphony of longing).
I started experimenting with the concept of a musical state machine, which harness the logic of Ableton's logic of looping to present users with an intuitive way to direct a musical set.
A multi-track looping interface experiment using Web-audio api.
The instruments were modeled in Blender, in low poly. It looks like a mere aesthetic decision, but this way is also compatible with lower end computers.
The initial idea was to be surrounded by an abstract scenery with floating instruments. For me, that the track has some relation with the real presence of music, so I built instead an abstract scenery representing a performance scenario.
The different musical states among which this experience switches, gives an idea of a story line. This is why I decided to add distinct decorations to each different musical state, and optional "ending" to this story, where the user is suggested about a reflection where this experience took place: inside a head.