SUBPROJECTS Tools for diversity

Theoretical, academic and scientific research, Technical development and Artistic development.

DETAILED DESCRIPTION OF CASE STUDIES/SUBPROJECTS RELATED TO EXPECTED OUTCOMES AND PROJECT ACTIVITIES

In Relation to the 4 main expected outcomes:

  1. New methodologies, platforms and networks for transdisciplinary research, creation, dissemination and learning. To develop new approaches for the understanding of expression, emotion and cognition – To generate a novel technocultural paradigm for communication and the arts, which foregrounds embodied knowledge and cognition – Academic research outlining a new field for the understanding of cultural diversity, in the domain of non-verbal communication, expression and emotions.
  2. Tools for understanding and raising awareness of cultural diversity
  3. Tools to foster embodied communication for a sustainable culture
  4. An interactive mobile architectural structure for outdoor spaces. Laboratory and observatory of diversity.

EXPECTED OUTCOME – Tools for understanding and raising awareness of cultural diversity.

  1. Disalignments (Reverso) – new movement techniques that highlight diversification of movement-expression through subtle deviations from usual movements.
  2. Microsexes (Reverso)- performance and prototype experimenting new kinds of embodiment with focus on sex-gender diversity and on diverse abilities.
  3. Illegible affects (Reverso – UNIGE) – challenging approaches to emotion analysis that focus on standardization and focusing instead on the indeterminate and ambiguous aspects of expression and emotion.
  4. Metabody Box/Motion Composer (Palindrome – IMM – Infomus) – an artistic platform and a technological device for people with disabilities in which interactive technologies are used to allow people with disabilities to generate musical sound through small movements of any body part.
  5. Affording Difference (Palindrome) – The project proposes the development and implementation of tools of interactive technology for different(ly-abled) bodies.
  6. Body is not what it used to be (Palindrome) – Interactive and inclusive dance piece, by and for persons with and without disabilities, with gesture/movement based technologies extending different bodies
  7. Metainterview (Palindrome) – performative installtion piece involving micromovement analysis of facial microgestures in a one to one situation
  8. Tiresias (Palindrome) – An interactive dance piece created for a person with disability and 2 dancers
  9. Performance of perception (Palindrome) – A dance performance and research about human perception with interactive technology.
  10. Touch matters (Palindrome) – This collaborative research explores various technologies from embodied sensors to the physical touch between two or more persons.
  11. Methodologies of diversity (Universidad de Valladolid – Associated Partner) – Developing a methodology to understand diversity and novelty of expression and interaction.

 


Illegible affects

Conception: Reverso – Jaime del Val
Software for analysis of movement expressive qualities: Casa Paganini-InfoMus Research Ventre, University of Genoa.

Recordings of movement qualities: K-Danse, Instituto Stocos.

Description:
This case study/subproject studies the computational analysis of emotion and movement for interaction design purposes from an artistic perspective informed by science and technology, focusing on ambiguity and indeterminacy of expression as a necessary, unavoidable and creative substrate of human non verbal communication.
Illegible Affects: application of Kohonen self-organizing neural network models for the analysis of “illegible” movement qualitaties.
This project, conceptualised by Jaime del Val, proposes to expand on one of the most fundamental areas of research of the Metabody project, namely the aspects of expression that while escaping quantitative or reductive modes of analysis, are crucial for creativity in communication. Illegible Affects proposes to raise a positive awareness of the indeterminate, ambiguous or illegible expressions, which enrich the spectrum of our communications making them more creative.
The project proposes to focus on the positive aspects of dissimilarity of movements, through real time machine learning systems that allow to generate maps of movement patterns with colour regions which will be clearer the more repetitive a pattern is, raising awareness of repetition, while the map will be richer in colours the more veried the pattenrs are. Once a map with clear regions has been generated one can blur these regions through performing gestures that don’t correspond to the patterns previously generated. The feedback of the application allows to focus on degrees of dissimilarity of movement on a contingent basis, not in relation to universal parameters but to the actual patterns generated here and now by the user. The map thus invites to explore gestural diversity and divergency as a positive value.
The application gives simultaneous feedback of dissimilarity, slowness, energy, fluidity and weight, avoiding emotional labels for the analysis of gesture while giving awareness of the multiplicity of qualities that a given system analyses all the time in our expressive gestures. In particular the slowness awareness operates only at a scale of extreme slowness, that resonates with the movement techniques developed in Metabody under the so called Disalignment techniques, where slowing down is proposed as a first gesture of resistance to control society and its constant acceleration of movements.
The application is clearly a response, partly ironic, to most of the research being done in the Big Data Culture of prediction and modulation of emotions and behaviours, trying to subvert this tendency by giving a positive value to indeterminacy and inviting to explore gestural diversity and divergency.
Purpose and relation with Metabody outcomes: The case study approaches one of the main issues proposed by the project: “to invert the habitual process undertaken in information technologies, so that instead of foregrounding meaning and function, we foreground the embodied expressive elements of communication, and the ways in which these exceed meaning structures. What new relational, cognitive and affective frameworks could be generated in this process through radically highlighting the openness of expressive qualities? How could this bring about an improvement for the lives of people who do not match the dominant structures of functionality and meaning, from disabled people to people from diverse cultural backgrounds, gender, or age groups?” “to develop new approaches for the understanding of expression, emotion and cognition that do not aim to classify patterns, which always involves a reductionism, but to understand processes of change and diversification; to understand the processes of affective and cognitive homogenisation in current globalisation; and to foreground the changing features of embodied expression that are foundational to cultural diversity, sustainability, and heritage.”

What has been done:

  • The case study was proposed by Jaime del Val in the Dresden 2013 Forum, Madrid 2015, and Paris 2015 Forum.
  • Exchanges between Jaime del Val and UNIGE-Infomus staff took shape during the Genoa Forum.
  • Recordings of movement for the development of novel analysis methods was done by Infomus, with guidance and proposal of K-Danse (Jean Marc Matos) and dancers provided by Instituto Stocos in the Madrid Forum 2014.
  • Presentations at Forum Madrid 2015 and Paris 2015.
  • Final developments till January 2016.

Results:

  • Software.
  • Artistic installation using smartphones as interaction devices – IMF 2015
  • Video.

Disalignments

Jaime del Val (Reverso) Muriel Romero (Instituto Stocos)

Description:
Disalignments is an artistic project that proposes the development of movement and choreographic techniques that generate an awareness of the standard gestures and choreographies that are distributed worldwide by information technologies, while allowing to disalign oneself from these choreographies accounting for diversity of expressions.

Purpose and relation with Metabody outcomes:
Develop movement techniques that raise awareness of homogenisation of expressions and facilitating to move away from that homogenisation through exploring new ways of movement and perception.

What has been done:

  • Presented at the Dresden and Genoa Forums.
  • Ongoing experimentation in collaboration between Reverso and Stocos in Madrid and with UAM.
  • Workshops in:
    • Madrid Forum 2014.
    • Amsterdam forum 2014.
    • Weimar Forum 2015.
    • Usa-Canada Forum 2015.
    • Madrid Forum 2015.
    • Pairs Forum 2015.
    • Colombia-Chile Forum 2015.

Disalignments (English).
Desalineamientos (Spanish).

Results:
Movement techniques related to the architectural structures of the Metabody project implemented in all workshops imparted by Jaime del Val in 2014 and 2015.


Microsexes 2.0

Reverso – Jaime del Val.

Description:
An artistic project that proposes new modes of perception and movement bringing together gender/sexual minorities and people with disabilities in which surveillance cameras are placed on the skin while the closeup images of the body captured by the cameras are projected life. It proposes new formats of one to one encounter, preparing the ground for the experiences afforded in the mobile architecture that focuses on involving the participant intimately so as to facilitate new perceptual and proprioceptive experiences.

Purpose and relation with Metabody outcomes:
The project proposes to challenge the dominant visual perceptions afforded by cameras generating a new awareness of the body that affects both the question of ability and of sex-gender difference. In this new scale of movement there is no clear possibility to measure ability, while the body becomes an amorphous landscape that challenges anatomical identification according to gender, sex or other criteria. The project generates a new awareness of difference and plurality as indeterminate, while it affords new modes of embodied perception and movement.

What has been done:

  • Testing, rehearsing, production.
  • Software for voice processing.
  • Wearable systems.
  • Performances.

Microsexes (English).
Microsexos (Spanish).

Results:


Methodologies of diversity

Universidad de Valladolid. – Alicia Peñalba.
IMM, Palindrome, Reverso.

Description:
Work on methodologies for identifying diversity of expression. Collaborating with Palidrome in installing the Motion Composer inside Museo Patio Herreriano, Facultad de Educación y Trabajo Social (Universidad de Valladolid), Centro Obregón de Parálisis Cerebral (ASPRONA), Centro El Pino (ASPRONA), Colegio Público Antonio Allue Morer, Residencia Asistida de Tercera Edad La Rubia, to compile tipologies of movement focusing on differences rather than similarities, from the audience interaction, to study kinds of interaction in people of diverse ages, genders, backgrounds, with and without disabilities. The proposal raises the question of how to use technologies to generate the immeasurable and unpredictable. It also proposes to work towards the final prototype on an architecture that encourages the expression of gestural difference.
Purpose and relation with Metabody outcomes: Research on methodologies to understand and identify diversity in embodied expressions and for facilitating the emergence of new expressions, movements and behaviours .

What has been done:

  • Introduced at the Genoa Forum 2014.
  • Provisional results presented at the Madrid Forum 2014.

Results:


MetaBody Box / Motion Composer

Palindrome Dance Company, IMM, UNIGE – Infomus, Instituto Stocos, Amber Platform, Universidad de Valladolid, Fundación Música abierta and external collaborators.

Description:
Metabody Box/Motion Composer is an artistic platform and a technological device for people with disabilities in which interactive technologies are used to allow people with disabilities to generate musical sound through small movements of any body part.
Purpose and relation with Metabody outcomes: It is one of the foreseen tools and methodologies for raising awareness of cultural diversity as well a for new embodied forms of communication that highlight diversity. The research and development of this subproject focuses on differently abled bodies (people with “disabilities”) thus providing one of the crucial aspects for the notion of cultural diversity and embodied communication: rather than forcing people with different abilities (disabilities) to subject themselves to the standard norms of communication, the tools provides new means of expression in which the gestures of people with disability acquire a full dimension of expressive quality, while affording new movement potentials.

Purpose and relation with Metabody outcomes:
It is one of the foreseen tools and methodologies for raising awareness of cultural diversity as well a for new embodied forms of communication that highlight diversity.

What has been done:
On-going research implies continuous tests with disabled people and dancers as well as technical research, between the partners and with external collaborators. Different stages of this artistic work were presented in the different Forums with creation workshops and public presentations .

Results:

  • Six different artistic-musical platforms develped by six composers that present six different interactive environments for people with disabilities with three different levels of motion-ability: standing, sitting, lying on a bed.
  • Software and hardware.
  • Presentation at the Forums in the form of workshops (free) lectures and performances.
  • Methodologies.
  • Papers for Metabody Journal and other publications:
    • Wechsler, R., Bergsland, A., Lavau, D.; Affording Difference : Different Bodies / Different Cultures / Different Expressions / Different Abilities, IMF CITATION FROM FINAL MetaBody Journal needed.
    • Bergsland, A. and Wechsler, R. (2016). Interaction design and use cases for MotionComposer, a device turning movement into music. SoundEffects – An Interdisciplinary Journal of Sound and Sound Experience, special Edition on: Sound and Listening in Healthcare and Therapy. Vol 5, No 1 (2016).
    • Dietz, J.; “MotionComposer….”; Unterstützte Kommunikation, Kongress 2015; ISAAC (International Society for Augmentative and Alternative Communication), Technische Universität, Dortmund, Germany, 9.2015.
    • Bergsland, A., “Aspects of digital affordances:Openness, skill and exploration “, Presentation at Affordance of Symbiosis – International Metabody Forum, Weimar March 2015.
    • Torres, R., “Musicoterapia, el poder curativo de la expresión artística interdisciplinar”, SALUD-Musicoterapia, November 2014.
    • Bergsland,A.; Wechsler,R.; “Composing Interactive Dance Pieces for the MotionComposer, a device for Persons with Disabilities”, Proceedings of NIME2015, BatonRouge LA, USA, 2015.
    • Nicole Strecker; “Tanz Dich Frei”, TANZ_Zeitschrift; Berlin, Mai 2015.
    • ALICIA PEÑALBA, MARÍA-JOSÉ VALLES, ELENA PARTESOTTI, ROSARIO CASTAÑÓN, MARÍA-ÁNGELES SEVILLANO; Types of interaction in the use of MotionComposer, a device that turns movement into sound; Proceedings of ICMEM – The International Conference on the Multimodal Experience of Music, University of Sheffield, England, 2015.
  • Body is not what it used to be – An interactive and inclusive dance piece with 6 dancers with and without disabilities.
  • Tiresias – An interactive dance piece created for a person with disability and 2 dancers. It uses the MBB to explore movement expression in persons with disabilities.

Metabody Box.
Affording Difference. 
Workshop Metabody Box.
Jaime del Val testing micromovements with Metabody Box. 
Jaime del Val testing Metabody Box. 


Affording Difference: Different Bodies, Different Cultures, Different Expressions, Different Abilities

Palindrome 
Additional researchers/contributors: Andreas Berglsand, Jaime del Val (Reverso), Josepha Dietz, Annika Dörr, Ekmel Ertan, Marcello Lussana, Pablo Palacio (INSTITUTO STOCOS), Alicia Penalba, Marije Baalman (STEIM), IMM.

Description:
Affording Difference, is a project within the MetaBody framework(1) to study:

  • Tools for understanding and raising an awareness of cultural diversity.
  • Tools for fostering embodied communication for a sustainable culture.

The project proposes the development and implementation of tools of interactive technology for different(ly-abled) bodies, challenging the notion of ability/disability and considering all bodies as different(ly-abled), while specifically paying attention to bodies that are usually considered disabled and affording them new tools of expression in which their difference acquires a positive value. Using motion tracking technology (including video-based and controller-based systems), dance and music, the questions the research poses include: How can we remove barriers to expression?
Technology tends to reduce gestural expression, how can we expand it — expand range of movement, range of expression? How can we promote a more positive awareness of difference, through disalignment(3) from normative conception of ability or intelligible expression? How do we generate affordances that invites deviant (alternative) behaviors which foster plurality and not homogenization?
What are the cultural differences in the perception of difference? How do these differences relate to acceptance and integration (inclusion)? How do different societies (including governmental and non-governmental organizations) approach inclusion?
Palindrome.

Purpose and relation with Metabody outcomes:

  • Elaboration of tools for embodied diversity.

What has been done:

  • Workshops and presentations in IMF Weimar, Madrid, Berkeley, Paris and research in Germany and around Europe.

Results:

  • Methodologies for building inclusive environments for people with disabilities.
  • Paper for Metabody Journal.

Metainterview

Palindrome.
Artistic Concepts: Delphine Lavau and Robert Wechsler.
Musical Composition: Pablo Palacio (Instituto Stocos).
Technical developments: Palindrome, Instituto Stocos (Pablo Palacio), Marcello Lussana.

Description:
Addressing the importance of non verbal communication and embodied expressions for cultural diversity as a fundamental form of cultural heritage that is not adequately taken into consideration, and which is being undermined by current information technologies, which induce unprecedented forms of homogenization of non verbal expressions while subjecting people to an increasing control, thus undermining fundamental freedoms.
Two chairs face each other a short distance apart. The visitor will experience a strange conversation with an interviewer. An installation-performance based on the idea of the confessional room in reality shows, the interviewer will investigate the visitor, look for his truth, challenging intimacy and free will. Visitor is engaged in a largely non-verbal interaction, virtual corporeality and bodily expression through facial gestures and touching. Each gesture or touch triggers a sound, environmental changes, room lighting changes, or color-light panels. This experience will be different for each visitor, guided by body-language.
Technologies: CCD Camera technology, PC+MAC computers with industrial frame-grabber, EyeCon motion tracking, SuperColider Music creation software Palindrome – MetaInterview.

Purpose and relation with Metabody outcomes:

  • Elaboration of the tools for embodied diversity as an artistic installation for the future architecture that elaborates some aspects of the Metabody philosophy such as how to promote novel expressions and raise awareness of micro expressions in a creative way, as well as on the ways of involving the audience in intimate encounters that transform perception.

What has been done:

  • Premiered at Medialab Prado, IMF Madrid, 22-24 July, 2015.

Results:

  • Artistic interactive and performative installation.
  • Paper for Metabody Journal.

Tiresias

Palindrome Dance Company e.V.
Choreography: Robert Wechsler.
Performers: Robert Wechsler, Annika Dörr and Thomas Mader.
Musical composition: Andreas Bergsland.

Description:
An interactive dance piece created for a person with disability and 2 dancers. It uses the MBB to explore movement expression in persons with severe disabilities. As a part of their work with persons of other abilities, Palindrome created a piece with persons of other abilities. In Greek mythology, Tiresias was a blind prophet of Thebes, famous for clairvoyance and for being transformed into a woman for seven years. Tiresias is an interactive dance piece exploring ranges of dynamic in expressive gesture – from small discrete finger movements, medium sized “conversational” gestures, to large energetic swipes and explosive outbursts – and how these movements can be interpreted sonically. The piece plays on the theoretical work of composer Francois Bayle’s dealing with the phenomenology of listening dealing with acousmatic music – music without visual sound sources. While Bayle maps out modes of musical sound/play using movement metaphors like “rub”, “pull”, “push” or “slide”, we turn this around and instead map out movement modes using sound. The sounds palette of the piece is based on hundreds of prerecorded sound particles. The sound particles make up a rich amalgam of sound objects, differing greatly in quality and feel and the degree to which they invite concrete associations and narratives. They are played purely by the dancers’ movements (no movement=no sound). A small discreet finger movement plays a single sample, but large gestures result in concatenations, building into chains and finally leading to dense clouds or roaring masses of sound.
Technologies
CCD Camera technology, PC+MAC computers with industrial frame-grabber.
EyeCon motion tracking, CSound Music creation software
Creation: October 2013
Length: 12 minutes
Tiresias.

Purpose and relation with Metabody outcomes:

  • Artistic elaboration of tools for people with disabilities.

What has been done:.

  • Premiered at IMF Dresden, November 17th, 2013.

Results:

  • Performance.

Performance der Wahrnehmung

Original Concept: Marcello Lussana.
Theoretical research: Marcello Lussana and Marlon Barrios Solano.
Choreography: Robert Wechsler.
Music composition: Oscar Palou.
Performers: Martina Nitz, Rebekka Böhme – In collaboration with Thikwa theater Berlin.

Description:
A dance performance and research about human perception with interactive technology.
The goal of this work is the creation of a dance performance with the use of interactive technologies, so that the performers have some control on the generated music. This work is part of a research on human perception and wants to analyze how the perception of movement is influenced by interactive technology and how it can be shown with dance.
Technologies: Motioncomposer and Motiontracking, Xbees, smartphones, accelerometer and gyroscope data, Closer, a wireless proximity and touch sensor, other touching and proximity systems.

Performance der Wahrnehmung.

Purpose and relation with Metabody outcomes:

  • Artistic elaboration of tools for persons with disabilities.

What has been done:

  • Theoretical research on perception and interactive dance performance for 2 dancers.

Results:

  • Inclusive performance for 2 dancers (1 in wheel-chair).

Touch Matters

Palindrome with Pablo Palacio (Instituto Stocos), Marije Baalman (Steim), Marcello Lussana, Jonathan Reus and Oscar Palou.

Description:
This collaborative research explores various technologies from embodied sensors to the physical touch between two or more persons.
Using electrodes and digital technology, the authors have been experimenting with human touch — the skin-to-skin contact between two or more persons — as a controller for media. Touch is obviously something humans care deeply about; a touch can say more than words. Touch is normally an intimate experience, mainly affecting those doing the touching, and yet by turning it into light, sound, etc., it can take on new dimensions; what is hidden and small, becomes large and even public. This paper reviews some of the work that was done on Touching as a part of MetaBody, touching on philosophical, technical, artistic and therapeutic implications.

Technologies:
Various technologies were explored including capacitor-based, induction, radio-induction, audio feedback and skin resistance. CSound, PureData Music creation software were used.
Touch Matters.

Purpose and relation with Metabody outcomes:
Elaboration of tools for embodied diversity.

What has been done:
The work undertaken occurred between April and December 2015, at the Madrid IMF, at the Stocos Studio in Madrid, and then finally in the form of two “Touch Summits” in Berlin — weekends in which all engineers and artists gathered and worked at our studio.

Results:
Technology and artistic development paper for Metabody Journal II.


Body is not what it used to be

Palindrome Dance Company, IMM, RemDans, Amber Platform.

Description:
This interactive and inclusive dance piece, by and for persons with and without disabilities, presented ongoing artistic researches and some of the implementations that focus on gesture/movement based technologies extending different bodies. While new technologies extend and change the bodies we preferred to make a special emphasis on the disabilities. We call it “different bodies” but not “disabled bodies”, since each single body has its own communication and performance capacities, which are not indeed subject of disability but divergence. Bodies’ future lays in new technologies. Technology is increasing the diversity of the bodies and blurring the borders of the body, as well as its perception.
Differently-abled people are encouraged to participate and very welcome. The technologies of body and movement in the space can only be discussed with the participation of diverse bodies. Technologies: CCD Camera technology, PC+MAC computers with industrial frame-grabber, EyeCon motion tracking, CSound Music creation software.
Palindrome – Salt.
Amber Plataform.

Purpose and relation with Metabody outcomes:
Elaboration of tools for embodied diversity.
This collaboration lead to the thematic construction of IMF Weimar “Affordances of symbiosis; rethinking interaction and the immeasurable potential of the body”.

What has been done:
Artistic research and Performance.

Results:
Performance, installation.