This is an ongoing project illustrating the extraction of higher level features from mechanomyogram sensors (MMG) conducting in EAVI. Biosignals are often hard to deal with since the collected data are often hard to interpret, redundant, and noisy. Our approach illustrated in the video is to consider muscle activity as behaving as a spring-mass system. In other words, the muscle is oscillating and its oscillations are damped. For example a smooth repetitive movement will make the muscle oscillating at a certain frequency. On the contrary, jerky movements involve high damping.
The idea of the experiment is to play the violin with smooth repetitive arm’s movements while transient motion will make that the violin does not sound properly.
In terms of modeling, the muscle is assimilated to a harmonic oscillatory system. Then we perform system identification. In other words, the method used identifies the parameters of the system (namely frequency, damping coefficient and offset) in realtime. These parameters are used to control the violin’s parameters (namely bow pressure and bow speed).
The method for motion dynamic extraction has been implemented as a C++ library and interfaced in Max/MSP and Pure Data. The sensor used is the Xth-sense developed by colleague Marco Donnarumma. The sound synthesis is Modalys from Ircam Centre Pompidou.
Note that the video reflects a working progress on motion analysis, further improvements will also be made on sound design.