SUBPROJECTS Tools for embodied communication

Theoretical, academic and scientific research , Technical development and Artistic development.

DETAILED DESCRIPTION OF CASE STUDIES/SUBPROJECTS RELATED TO EXPECTED OUTCOMES AND PROJECT ACTIVITIES

In Relation to the 4 main expected outcomes:

  1. New methodologies, platforms and networks for transdisciplinary research, creation, dissemination and learning. To develop new approaches for the understanding of expression, emotion and cognition – To generate a novel technocultural paradigm for communication and the arts, which foregrounds embodied knowledge and cognition – Academic research outlining a new field for the understanding of cultural diversity, in the domain of non-verbal communication, expression and emotions.
  2. Tools for understanding and raising awareness of cultural diversity
  3. Tools to foster embodied communication for a sustainable culture
  4. An interactive mobile architectural structure for outdoor spaces. Laboratory and observatory of diversity.

EXPECTED OUTCOME – Tools for fostering embodied communication for a sustainable culture.

  1. Amorphogenesis (Reverso) – digital prototype for future architecture
  2. Bodynet (TMA – Reverso) – the first social network based on non verbal communication
  3. Emergent Proprioceptions (Reverso) – generating new perceptions through sensors on the body
  4. Video streaming and real-time analysis using Android smartphones (Infomus-UNIGE) – tools for embodied interaction.
  5. Gesture analysis based on IMU sensors (Infomus-UNIGE) – tools for embodied interaction.
  6. Development of an EyesWeb patch for the multimodal recordings of dancers (Infomus-UNIGE) – tools for embodied interaction.
  7. Dissappearing Dancer (Infomus-UNIGE) – tools for embodied interaction.
  8. Definition of a vocabulary of movements defined by Jean-Marc Matos (Infomus-UNIGE – KDanse) – tools for embodied interaction.
  9. Several developments(Infomus-UNIGE) – Expressive movements analysis of different situations – tools for embodied interaction.
  10. Fleshwaves (TMA) – performance experimenting new kinds of embodiment
  11. Emoticam (TMA) – performance experimenting new kinds of embodiment
  12. Holostage (TMA) – telematic performance and prototype
  13. Victory over the sun (Brunel – Dap_lab) – experimenting with wearables and design
  14. Metakimospheres (Brunel – Dap_lab) – Metakimospheres are kinetic atmospheres created for dancers and receivers exploring an intimate sounding architecture.
  15. Several designs for interactive performance (Brunel – Dap_lab): BeakHandSpeakerSpeakerVeilConductiveCoatKaidagaraDressKeplerAccordionDrag Object
  16. Sense of the Body (STEIM) – exploring wireless sensors on the body
  17. Softspeakers and sounding textiles(STEIM) – development of distributed embedded speakers in textiles
  18. E-textiles (STEIM) – reserach on e-textiles for embodied interaction
  19. OnTouch (Steim) – research on tactile feedback technologies for embodied interaction

Amorphogenesis

REVERSO – Jaime del Val.
Dtr-lab Dieter Vandoren (Associated partner), TUDelft, Marije Ballman (Steim).

Description:
Amorphogenesis is an instrument and installation for digital interactive/intra-active architecture in which the movement of bodies transforms amorphous digital meshes in neverending emergence. Amorphogenesis is also a philosophical concept by Jaime del Val that points to an account of movement as not subjected to form nor poiting to it.

Involvement of the partners and associates:

  • Reverso has provided the concept, artistic development and double cylinder of screens for projection system.
  • Dtr-Lab – Dieter Vandoren has provided work on the programming and projection system based on four projections.
  • TUdelft has provided test and rehearsal studio and the work of Jia Rey Chang on designing the digital architectures.
  • Steim has provided the wireless sensors.

Purpose and relation with Metabody outcomes: Amorphogenesis is a digital prototype for the future architure pavillion touring europe as well as a tool for fostering embodied communication. Amorphogenesis proposes the use of wearable technologies (sensors on the body) to develop a new experience of space as dynamic reality that transforms with movement.

What has been done:

Amorphogenesis (English).
Amorfogénesis (Spanish).

Results:

  • Software for Digital interactive architectures and sound.
  • Integration of Sensors on the body for interaction.
  • Projection structures and systems.
  • Ultraportable Environment integrated in the metatopia architectures.
  • Performances and installations.

Amorphogenesis 2.0 – First tests June 2014.
Amorphogenesis – Madrid 2014.
Amorphogenesis – Microsexes.
Amorphogenesis – in Montreal – Hexagram – April 2015.
Amorphogenesis tests in Montreal at Concordia University IMF 2015.


Bodynet

Reverso & Trans-Media-Akademie Hellerau. 

Description:
Prototype for the first social network based solely on non-verbal communication.

Purpose and relation with Metabody outcomes:
This is one of the main foreseen outcomes of the project, related both to “tools for embodied communication” and to the final architecture. The Bodynet subproject specifically challenges the “disembodied” charachter of current social networks, that standardise expressions and facilitate social control.

What has been done:
Preliminary research. Discussions on the forthcoming development of this outcome have happened at the Fora.

Results:
Partial implementation as OSC network, which implies using the OSC protocol for transmission of data without use of the internet, directly between devices, at alocal level but at distance (wirlessly), bypassing the Big Data control systems, and proposing hybrid of local-telematic interaction based on the non-standard and non-linear mapping of data of embodied interaction. This protocl is the main software intercommunication protocol used in the Metabody project to interconnect in a plural way all the different technological platforms without homogenising the software and hardware used.


Emergent Proprioceptions

Jaime del Val – REVERSO & STEIM.

Description:
Research into new perceptions of movement through new interfaces using wireless sensors on the body.

Purpose and relation with Metabody outcomes:
Connected with STEIM’s project on wireless sensors, Reverso aims to explore how the use of such sensors may transform radically the body’s perception of itself.

What has been done:

  • Preliminary research.
  • Presentation at the Dresden Forum 2013.

Results:

  • Preliminary research.

Video streaming and real-time analysis using Android smartphones

UNIGE- Infomus.
Technical direction:
Corrado Canepa.

Description:
One of the conceptual aspects in MetaBody is to focus on different views and perspectives on the analysis of the body in both individual as well as group performances. As a technical contribute in this direction, we designed and implemented an Android app for smartphones able to capture video in real-time from the onboard video-camera, and to stream the captured video in real-time to an EyesWeb server. This included the development of a new EyesWeb software module for receiving a video stream from smartphones: this is now added to the library of software modules in EyesWeb. This development enables artists to design real-time video analysis and processing from one or more smartphones.

Purpose and relation with Metabody outcomes:
Developments of tools for embodied communication and for the mobile architecture.

What has been done:
Designed and implementation of an EyesWeb patch (application) for real-time optical flow analysis, using the video signals received by smartphones streaming.
Artistic use: “Monster-Metabody” (K.Danse) project, to collect videos from single audience members of artists on stage for having personal views of the performance from the audience on stage (at the center of the labyrinth). In general, collect individual views, perspectives on gesture communication, to explore ambiguous views and perspectives on everyday gesture. The dynamic interactive architectures and sensitive spaces are also a target for this development, for example to collect personalized views during an exploration of a mobile interactive architecture.
This software development is now part of the free distribution of the new release of the EyesWeb platform.

Results:

  • Software
  • Artistic uses in Monster subproject

EyesWeb video streaming and real-time analysis using smartphones.


Gesture analysis based on IMU sensors (including sensors on smartphones)

UNIGE- Infomus.
Technical direction:
Corrado Canepa.

Description:
Analysis of rhythmic, periodic movement patterns, using the on-body sensors available on smartphones. We developed new EyesWeb modules for the real-time analysis of BPM (beat per minute), energy, and other movement features starting from 3-axis accelerometers of an Android smartphone.

Purpose and relation with Metabody outcomes:
Developments of tools for embodied communication and for the mobile architecture

What has been done:
Developed an EyesWeb application connected to wireless smartphones, capable to segment movement into gestures, and to recognize individual gesture typologies, according to energy, bpm and speed.
Artistic use: this movement analysis techniques can be used for example in environments where videocameras cannot be adopted to measure gesture, and in particular in the Metabody dynamic architecture and in the originally conceived “pavilion”. This information can be exploited for the real-time control of music and visual signals, related to the qualities of gesture. The “Illegible Affect” experiments to explore what can be recognized and what cannot from a wearable was a case study where the system has been evaluated and experimented: to use ambiguous measurements for sonifications and visual mapping was left to artistic experimentation and explorative studies.

Results:

Interactive music retrieval by movement features using mobile phone.


Development of an EyesWeb patch for the multimodal recordings of dancers using Kinect mocap, accelerometers on a weared smartphone, or minibee from STEIM, Amsterdam.

UNIGE- Infomus.
Technical direction:
Corrado Canepa.

Description:
Development of an EyesWeb patch for the multimodal recordings of dancers using Kinect mocap, accelerometers on a weared smartphone, or minibee from STEIM, Amsterdam.

Purpose and relation with Metabody outcomes:
Developments of tools for embodied communication and for the mobile architecture.

What has been done:
Multimodal synchronized recordings were done at workshop in Madrid (July 2014), and at STEIM December 2014.
A preliminary experimentation was made with Jean-Marc Matos, Marcello Lussana and dancer Roberta Messa in Genoa (June 2014) as to compare the recordings of “elegant, continuous, fluid movement” informed by the Theseus hero character of “Monster” and “deformed, monstrous, broken up movement” informed by the Minotaur’s character.
For the Recordings made in Madrid (July 2014) K. Danse provided a series of instructions for the creation of movement, first sent to various dancers, before being experimented live and recorded.
5 out of a set of 19 Units were prepared with simple instructions, open to much diversified interpretations and set to be experienced by any moving body (see the attached full description of the 5 Units : Heaviness, Contracted-Extended, Without direction by divergence of information, Run-up and Fall). For the recordings in Amsterdam (December 2014) these clusters of combined qualitative units were broken up into more simple quality instructions (as derived from the Laban approach), improvised by dancer Miri Lee and sonified by Pablo Palacio and Marije Baalman.
This is a useful tool to obtain multimodal movement datasets of dance and movement, for further study, analysis, and support to artistic projects in MetaBody.
These EyesWeb software modules were also released to some MetaBody partners for artistic performances.

Results:


Disappearing Dancer

UNIGE- Infomus. 
Technical direction:
Corrado Canepa.

Description:
A Casa Paganini-InfoMus proposal based on the metaphor of a virtual mirror where one or more dancers can emerge or disappear, according to specific “mirroring qualities”.

Purpose and relation with Metabody outcomes:
Developments of tools for embodied communication and for the mobile architecture

What has been done:
We developed two EyesWeb applications:
Disappearing Dancer 1: The dancer disappears when she does repeated predictable movements (based on real-time analysis of the recurrence of the gesture – Recurrence Quantification Analysis).

Disappearing Dancer 2 : only sudden jerky movements cause the dancer visible, fluid movements cause the dancer to disappear (based on real time analysis of Impulsivity in the gesture)
The first versions of these two EyesWeb patches were presented at the event in Madrid (July 2014), then they were refined for Steim event (Dec 2014). New versions developed in 2015 are based on new algorithms of Recurrence Quantification Analysis and on refinements of movement qualities.

Results:

Demo: the dancer disappears if he does repetitive and recurrent movements.
Disappearing dance – demo: the dancer disappears/appears if she does fluid/impulsive movements.


Definition of a vocabulary of movements defined by Jean-Marc Matos (K.Danse) and recorded in Madrid, based on accelerometers and Kinect.

UNIGE- Infomus.
Technical direction:
Corrado Canepa.
Recordings under guidance of K-Danse with collaboration of Stocos.

Description:
Definition of a vocabulary of movements defined by Jean-Marc Matos (K.Danse) and recorded in Madrid, based on accelerometers and Kinect.

Purpose and relation with Metabody outcomes:
Developments of tools for embodied communication and for the mobile architecture.

What has been done:
This work includes an EyesWeb application to extract expressive features from the movement of dancers and video data captured with accelerometers on smartphones.
Jean-Marc Matos has brought into Metabody a body of previous research made on SUTD (Semantic Units of Time for Dance) as transposed from the field of electronic music into dance composition. This has given birth to the devising of a map of 19 qualitative movement clusters (or units) containing each a mix of quality factors. The intricacy put forward has demonstrated a rather complex approach which challenges the computer analysis due to the diversity of interpretation by dancers providing data difficult to decipher. These qualitative movement units provide a different approach, away from The Laban oriented one, able to encompass a larger spectrum of expressive movements and yet more difficult to be identified. The scientific and technological challenge is enhanced by the fact that these combined qualitative units can be used for micro-dramaturgies per se, and not just as a set of different expressive movements. An even higher level of analysis for complex visual and sonic mapping is at stake in this context.

Results:
Software libraries within the Eyesweb platform. 
• artistic uses in other subprojects particularly Illegible Affects and Monster.


Various developments by InfoMus

UNIGE- InfoMus. 
Technical direction:
Corrado Canepa.

Other developments by InfoMus:

  • Analysis of different situations defined with Jean Marc Matos and recorded in Madrid, based on accelerometers and Kinect. This work includes an EyesWeb patch to extract expressive features from the movement of dancers and video data captured with accelerometers on smartphones
  • Training of machine learning models for the recognition of different situations of motion (quick, heavy, burst, falling) and events (jumps, pirouettes) from mocap data and expressive features.
    Use in MetaBody: first prototype experimented at Steim Workshop
    The database of dance recordings were not suitable for a reliable training of machine learning modules. We are open to evaluate with partners possible exploitations.
  • Development of an EyesWeb patch for the recording and feature extraction in real-time of dancers in Toulouse (January 2015), based on Kinect or Kinect2, tested at Steim and released to Jean-Marc Matos for the registration of professional dancers
  • Remote support for EyesWeb installation, support for the use of the IDS camera in EyesWeb, support to use the analysis patches and of the recording tool to personnel in Toulouse.
    Use in MetaBody: support to Monster artistic project in Toulouse
  • For Pablo Palacio Instituto Stocos and Daniel Bisig (February and March 2015):
    • development of EyesWeb blocks to support computing and sending silhouette contour via OSC (OpenSoundControl). More precisely:
      • added support to pack Point2D and List datatypes to a network packet
      • added support to pack Contour2D datatype to a network packet
      • added support for OSC blob datatype (to send binary data over OSC)
    • Development of EyesWeb patches to send/receive silhouette contour via OSC
    • Development of an EyesWeb patch to compute the silhouette contour from the DepthMap (instead of the binary silhouette)
    • Development of an EyesWeb patch to analyze the movement of the simulated creatures. First prototype developed and shown during the Steim workshop. Further refinement for the density index provided to Daniel.
    • Support about using Kinect2 in EyesWeb and changes with respect to Kinect1
    • Development of an EyesWeb patch to integrate the human (mass-spring) model developed by Daniel Bisig with the EyesWeb tracking system, through a network connection. First prototype developed and shown during the Steim workshop. The work has continued by integrating the Bisig model in a new EyesWeb block, in order to avoid the network latency and support further use-cases (e.g., to train the model on the data from a specific body, use the model for movement prediction, etc).

Use in MetaBody: support to Phantom Limb artistic project.


Holostage

Trans-Media-Akademie Hellerau.
Jo Siamon Salich (concept, project direction, production design), Frieder Weiß (software development), Matthias Härtig (programming), Johanna Roggan (dance performer), Katharina Groß (documentation), Thomas Dumke (project coordination).

Description:
Telematic interactive performance-installation HOLOstage is an extension to the idea of the European Tele-Plateaus project (ETP) by the Trans-Media-Akademie Hellerau (TMA). The actual background of the HOLOstage project is the 3D representation of the dancers who not only transmit position data and the silhouette of the virtual performers but in this special case their entire joint coordinate data, either with the help of motion capture in the camera interface or camera systems which PrimeSense depth sensors enable a three-dimensional mapping. In this virtual resolution with real-time transmission the coordinates of the entire body allow to decrypt the dancers as 3D characters.
In the implementation of the aesthetics, it is a wireframe model, which is representative then presented to the presence of the dancers and makes present in a new form. In the model representation this wireframe includes the abstraction of a virtual appearance.
To really stand out as three-dimensional interaction objects for the real dancers available the 3D objects or virtual avatars, which are proxy objects for the performers on the holographic stage, of course need an extension in the visualization then.
Implementation of this new form of three-dimensional presence is the 360° holographic projection, an interactive Cheoptics system with real-time transmission.
For the final performance situation we intend that the real-dancer interacts with three or four avatars and makes contact via image and sound modulation.
When improvising with these otherplace performers and exploring interactive constellations she mediates the simulated presence of virtual dancers directly, so the own creative dance movements happen reactively in a new way.
What is an interaction in a virtual environment when it can give no extended self-experience for the protagonist and no extended immersive experience will arise for the spectators? HOLOstage is a production of the Trans-Media-Akademie Hellerau, which is funded in a METABODY project development and already in the first stage shows partnerships.

Purpose and relation with Metabody outcomes:
To test possibilities of immersive “holistic” environments working on the grounds of new embodied telematic relations, as prototype for the Bodynet sub-project.

What has been done:

  • Research in Dresden in different sessions in 2014.
  • Introduction in the Madrid Forum 2014.
  • Presentation in Amsterdam and Weimar forums.
  • Performances in Dresden.

Results:


Emoticam

Trans-Media-Akademie Hellerau.
Johanna Roggan / Lorenz Ebersbach.

Description:
The artist and the audience wear a micro-controller propped pulse camera at their own figure. During the performance we will create rooms which address to various emotions. The acquired pictures of audience and performer will be visible through a projection and therefore will as feedback become again a part of the performative installation. View limiting objects should stimulate the audience to move itself through the installation. So it will become part of a constantly changing art work which removes the borders between the acting artist and the viewer. The audience is encouraged to self-creation and itself gives the input for a collective perception from the inside.

Purpose and relation with Metabody outcomes:
Experiment new forms of embodied interaction with audience.

What has been done:

  • Development of artistic and technical research.
  • Presentation in form of performance at the Dresden Forum.

Results:

  • Presentation in form of performance at the Dresden Forum 2013.

Fleshwaves

Conception and direction:
Isabelle Choiniere – Corps Indice.
Isabelle Choinière, Audrey-Anne Bouchard, Ricardo Dal Farra.

Description:
»Flesh waves« is an augmented reality performance/installation exploring the relationships arising from the interaction between the body in movement and technologies of sound spatialization in real-time and lighting. Under a faint glow appears a three-dimensional human sculpture: five women, bodies intertwined, form a Collective Body. Transported by the chant of their own murmurs, they slowly carry one another into a continuous wave. Bodies and breaths entangle, the women are simultaneously five bodies and one same mass, five voices and one choir. From the different dynamics of movement emerges an organic soundscape. Acting as the sixth dancer of the piece, the sound travels all around the spectators, integrating them in the live sound form. This immersion provokes a loss of bearings because the audience finds itself in the flesh: spectators are confronted to the extremely intimate. A poetic journey inside a hyper-feminine form characterized by sculptural eroticism, »Flesh waves« invites its performers and its spectators to reinvent together their sensorial experience of performance.

Purpose and relation with Metabody outcomes:
Experiment new forms of embodied interaction with audience.

What has been done:
Production of an artistic work of Isabelle Choiniere – Corps Indice – presented at the Dresden Forum in the form of performance and through lectures. This work of the guest artist Isabelle Choiniere was proposed as the core of investigation on embodied performance practices of the Dresden Forum.

Results:
Artistic work premiered at the Dresden Forum 2013.


for the time being [Victory over the Sun]

Dap_Lab – Johannes Birringer & Michèle Danjoux

Description:
Choreosonic Performance “for the time being” explores the sound of movement in the premiere of a new stage work inspired by the Russian Futurist opera Victory over the Sun (1913) and its fantastical visual designs. Nearly a hundred years ago, the Futurists collaborated on an eccentric vision of a society to come, based on revolutionary fervor of the time. The DAP-Lab’s choreosonic performance, both comic and unsettling in places, is an intimate piece that looks at our current precarious existence in a world full of outworn clichés of revolutions. Featuring dancers and musicians, the work incorporates body-worn-technologies where the structure of the wearable has been developed alongside its interactive and sound generating potential for gestural performance. The artistic premise of the design is to make the activation and presence of sound very visual and sensual to both the performer and the viewer.

Purpose and relation with Metabody outcomes:
Artistic and technical research into wearables.

What has been done:

  • Rehearsals and experimentation in Brunel, London, with DAP-Lab member Michèle Danjoux, and other collaborators.
  • On-going research development with Metabody partners
    (Hyperbody/TU Delft; Stocos; STEIM; Marcello Lussana; Reverso).
  • New networking relationship and cooperation with the London-based Stitch’ n Bitch group, a workshop for fashion designers, wearables artists, techno/textile and interaction artists, architects and sound artists, with meetings arranged by Camille Baker at London’s Watermans Art Center and the V&A.

Results:

  • Research on wearable and choreosonic concepts and performance; exploration of immersive and dynamic architectures for performance participation – performed outside Metabody in London, Sadler’s Wells, in 2014.

Metakimospheres

Dap_Lab – Johannes Birringer & Michèle Danjoux.
Collaboration with Hyperbody, Stocos, Reverso, Marcello Lussana.

Description:
Metakimosphere no. 1 / Metakimosphere no. 2 /
Metakimospheres are kinetic atmospheres created for dancers and receivers exploring an intimate sounding architecture which envelops and acts as a suspended transparent veil and encumbrance – hiding-revealing, allowing light and “graphic writing” to flow through, affording variable tactile orientations, colors, tones, shapes and positions, bodies forming a single body inside a cocoon-like changing gauze texture or physical-mobile interface. First tested in London in early 2015, the kimosphere not only reflects on the mediating conditions of conception as such – how we can think such a wearable architecture – but works through energies of tactile, breathing atmosphere (known in Japanese philosophy as “ki” and “aidagara,” and in Chinese cosmology as “Qi”) that are an “in-between“ phenomenon, generating fluidity between subject and object which the DAP-Lab design envisions as a kind of inside-inside architecture that can also become an inside-outside wearable. Inside-outside sound emission and different sonic contours and material behaviors (of the LOOP environment built by Hyperbody partners) mobilize new possibilities attached to the Loop Skin and yet removeable from it, challenging the visitors’ ideas of sensing the processual capacities for aesthetic work.

This project for Metabody has been crrated by DAP-Lab in cooperation with LOOP/Hyperbody Team 6 [TU Delft] working with moveable architecture and wearable space choreography. Presentation includes prototype development of designer Michèle Danjoux’s “BeakHandSpeaker,” “SpeakerVeil,” “ConductiveCoat,” and “KaidagaraDress.” Additional interfaces were planned with Jaime del Val’s metakinespheres/Metadress development, and del Val was invited to a workshop in London in late October 2015.

The DAP-Lab then c ontinued to prepare its third installment of the “kinetic atmosphere” with new conceptual performance-costumes interlinked to architectural kinetic states. These phenomena of presencing “dance” as atmospheres in space that can be felt, heard, and experiences through all senses, visual and non-visual, are being tested under the name “metakimosphere”. Initial kinespheric tests, proposed by Jaime del Val in Amsterdam, were elaborated by Azzie McCutcheon and the lab team in London (spring 20145). Azzie was joined by Martina Reynolds for test development, material construction, conceptual-choreographic evolution, along with Helenna Ren and Yoko Ishiguro, of the physical kimosphere-structure, as well as joined by Chris Bishop, Seeta Indrani and Cameron McKirdy for the software programming, the live kinect camera interface (Bishop) and generative algorithms written by McKirdy. The concept for an immersive installation was developed over the following weeks and months, and the second installment occurred during IMF Madrid (July 2015), where Michèle Danjoux and Johannes Birringer directed a new phase of research, joined by Jonathan Reus and Miri Lee, as well as a collaborating architecture team from Hyperbody, TU Delft. In late 2015, the expanded DAP-Lab team began building the third installment of the kimosphere, to be premiered April, 4-9, 2016 at the IMF London, which will be held as planned (even after the termination of the Metabody grant period in January 2016).
Metabody Archive 1.
Metabody Archive 3.
Kimospheres.
Youtube.

Purpose and relation with Metabody outcomes:

  • Immersive and performative environment for the future architecture.
  • Intra-active performance in collaboration with partners.

What has been done:

  • Rehearsals prior to and performance in Metabody Forum Madrid 2015.

Results:

  • Performance in Metabody Forum Madrid 2015.
  • Film produced of the kimosphere construction and collaboratio with Hyperbody / TU Delft architects of the {/S}caring-ami team (Anisa Nachett, Alessandro Giacomelli, Giulio Mariano, Yizhe Guo, Xiangting Meng). Presented at Metabody Forum, LeCube, Paris, October 2015.
  • Film on metakimosperes presented at Research Seminar Series, Brunel University London, November 2015;

BeakHandSpeaker

Dap_Lab – Johannes Birringer & Michèle Danjoux.
BeakHandSpeaker, designed and created for METATOPE exhibition space at Medialab Prado, Madrid, July 2015. Design by Michèle Danjoux. First tested at IMF Amsterdam, STEIM, December 2014 © DAP-Lab 2015.

Additional experiments conducted to make BeakHandSpeaker wireless; with Marcello Lussana, Medialab Prado, Madrid, July 2015.

BeakHandSpeaker2, new prototype designed and created for London IMF, 2016, first modeled in Madrid, July 2015 and Paris, October 2015. Design by Michèle Danjoux. Design production collaboration with Nicolo Merendino (STEIM) © DAP-Lab 2016.


SpeakerVeil

Dap_Lab – Johannes Birringer & Michèle Danjoux.

SpeakerVeil, designed and created for METATOPE exhibition space at Medialab Prado, Madrid, July 2015. Design by Johannes Birringer. First tested DAP-Lab, Artaud Center, Brunel University, March 2015. Redesigned Prototype for Muriel Romero created by Michèle Danjoux, IMF Madrid © DAP-Lab 2015.


ConductiveCoat

Dap_Lab – Johannes Birringer & Michèle Danjoux.

ConductiveCoat, designed and created for METATOPE exhibition space at Medialab Prado, Madrid, July 2015. Design by Michèle Danjoux. First tested at e-textiles workshop and at IMF Amsterdam (STEIM) October- December 2014 © DAP-Lab 2015.
ConductiveDress design under development with new materials for two further dresses, for IMF London 2016.


KaidagaraDress

Dap_Lab –Johannes Birringer & Michèle Danjoux.

KaidagaraDress, designed and created for METATOPE exhibition space at Medialab Prado, Madrid, July 2015. Design by Michèle Danjoux. © DAP-Lab 2015.


Kepler

Dap_Lab – Johannes Birringer & Michèle Danjoux.

Kepler , designed and created for METATOPE exhibition space at Medialab Prado, Madrid, July 2015. Design by Michèle Danjoux. Electronics and sound technics by Marije Baalman © DAP-Lab 2015.
Presented at LeCube exhibition, Paris, October 2015.


Accordion

Dap_Lab –Johannes Birringer & Michèle Danjoux.

Accordion, designed and created for METATOPE exhibition space at Medialab Prado, Madrid, July 2015. Design by Michèle Danjoux. Electronics and sound technics by Marije Baalman © DAP-Lab 2015.
Presented at LeCube exhibition, Paris, October 2015. Redesigned for IMF London 2016.


Drag Object

Dap_Lab – Johannes Birringer & Michèle Danjoux.

Sound-drag-object under development 2015-16, design by Michèle Danjoux. Electronics and sound technics by Neal Spowage © DAP-Lab 2016.


Sense of the Body

Marije Baalman, Bas Kooiker – STEIM, Reverso, K-Danse, Infomus.

Description:
Application of wireless sensor interfaces for the body, and development of data processing units for mapping this sensor data to sound, video and light.

Purpose and relation with Metabody outcomes:
wireless sensors will be a crucial technology for the development of the architecture and all the tools for new embodied expressions, including wearables and the Bodynet project.

What has been done:
Meetings and ongoing research between Steim and other coorganizers as well as with local collaborators, involving local artists and specialists.

Results:

  • Software and hardware for different uses by the other partners and by STEIM.
  • Artistic use within Amorphogenesis (Reverso/Dieter Vandoren) focussing on slow movement and sublte, indirect mapping to visuals and sound.
  • Artistic exploration with K-Danse focussing on fusion of sensor data based on activity levels of different parts of the body.
  • Metacusion – with K-Danse.
  • Placement of the sensors on the Metatopes (Reverso) to track movements of the extended and shared body, rather than the individual human body.
  • Integration of wireless sensing units with software from Infomus.
  • Further subproject on etextiles.
  • Online publication of tools on Sensestage Metabody Data and Sensestage OSC Gesture Classifier
  • Publication in the second METABODY journal

E-Textiles, wearables, movement and sound

Marije Baalman – STEIM, with local collaborators, and DAPLab (Michele Danjoux).

Description:
Research into electronic textiles in order to develop wearable sensing and actuating interfaces that can be used in performance in which the body plays a central role. This is partly an extension/specialisation of Sense of the Body, focusing on the wearability of sensors.

Purpose and relation with Metabody outcomes:
Wearable sensing and flexible architecture ask for different methods of sensing than rigid electronics can provide. Therefor we turned to electronic textiles (e-textiles) to look for sensing that can easily be made part of costumes to be worn on the body or flexible structures.

What has been done:

  • Research meetings with local collaborators and invited e-textile experts, ongoing research, and integration of outcomes of research meetings in the collaborative meetings at the METABODY forums.

Results:

  • Software and hardware for different uses by the other partners and by STEIM.
  • Artistic prototypes, such as the Malbody narrative and environment.
  • Haptic feedback for the Phantom Limb project.
  • Further subprojects: on touch and soft speakers.
  • Video documentation of research on STEIM’s Vimeo channel.

Soft speakers – sounding textiles – customized listening

STEIM – Marije Baalman, “Chi Ha Ucciso il Conte?” – Nicoló Merendino – Alberto Boem – Tijmen Lohmeijer.

Description:
Distributed sound reproduction with custom made speakers – exploring the concept of soft speakers – both in its material, as in the loudness of sound reproduction. We explore the properties of the materials – the speaker is not a separate entity, but the malleable shape of the object determines the overall sound quality. The speakers are connected to small embedded microcontrollers or computers that synthesize and produce sound – thus moving away from centralized audio production and towards distributed sound production. This subproject came out of the work on e-textiles as we found ways of creating loudspeakers and microphones with textiles.

Purpose and relation with Metabody outcomes:
Listening is a huge factor in how we move our bodies through space, changing the method of audio reproduction incites new ways of perceiving sound, but also movement through space. Given the requirements of flexible architecture, soft speakers can be embedded into flexible structures, or into objects.

What has been done:
Ongoing research at STEIM, and collaboration with HyperBody integrating sofspeakers into the Skarigami structure. Integration of outcomes of research meetings in the collaborative meetings at the METABODY forums.

Results:

  • Software and hardware for different uses by the other partners and by STEIM.
  • Video documentation of research on STEIM’s video channel.
  • Publication in second METABODY journal

Touch

STEIM – Marije Baalman; Palindrome – Marcello Lussana.

Description:
In everyday life, multitouch is a common mode of interacting with the flat surfaces of mobile phones and tablets. Rather than interacting with flat objects, we were interested in bodies touching each other, or touching objects with non-flat surfaces. We are also interested in emphasizing the subtle nuances of touching, rather than touching with a purpose of control.

Purpose and relation with Metabody outcomes:
Touch as an intimate, non-verbal gesture between two bodies. Also distance (which can be measured with the same technology) between two or more bodies is important in non-verbal communication.

What has been done:
Research meetings, ongoing development.

Results:

  • Scientific evaluation of different types of measuring touch and artistic evaluation of implications on working with these technologies.
  • Publication in second METABODY journal.