ISTITUTO DI SCIENZE E TECNOLOGIE DELLA COGNIZIONE Piero Cosi SEZIONE DI PADOVA - FONETICA E DIALETTOLOGIA Via Martiri della libertà, 2 – adova (Italy) - www: Facial Animation Geometry Manipulation Based on a Pseudo Muscle Model Lucia an Italian Talking Head Copyright, 2009 © ISTC-SPFD-CNR Incontro con Frank H. Guenther Director of CNS Speech Lab, Boston University, Boston MA "Neurolinguistica: produzione e percezione del parlato e controllo senso motorio" Martedì 22 settembre ore 10,30 Sala delle Statue, Palazzo Rospigliosi Via XXIV Maggio, 43 - Roma
Copyright, 2009 © ISTC-SPFD-CNR ISTC Applications SONIC SPHINX CLSU Toolkit ILT – Italian Literacy Tutor
Outline Motivation Who is working on similar … stuff? Movie Companies & Studios Animation Software Companies Research Institutes and Universities ISTC-SPFD – LUCIA & FESTIVAL Emotions (Audio & Video) and….what about the future!?
Motivation human-like conversational agents ( Virtual Characters, Computer Characters, Virtual Agents, Conversational Agents, Interface Agents…) human-like characters they interact naturally, using facial gestures, eye contact, speech patterns, posture and gesture, just like we might use them when conversing with other people one might see them in computer games or in the media users can interact and cooperate with them to complete a shared task
Soggetti Coinvolti RICERCA (modelli fisici) LUCIA GRETA BALDI ecc. PC, web, telefonia,... animazione Movie Companies & Studios Games ecc.
LUCIA a new emotive/expressive Italian talking head
ELITE System ELITE Video Processor (100Hz) A / D (16KHz) A/D Articulatory signals Acoustic signals
Track Animation example
Facial Animation Facial Modelling / Animation Geometry manipulations Parametrization Pseudo muscle model Wrinkle generation Free form deformation Spline model Pure vector based model
Facial Animation & TTS Input Text FESTIVAL Italian TTS segmental/lexical/semantic information AVengine ( TXT2Animation ) FAP WAV LUCIA
emotions and expressions universal expressions (fear)
Audiovisual Emotive/Expressive Speech Lucia neutraljoysadness disgustfear
Emotion Player Copyright, 2007 © ISTC-SPFD-CNR
ILT – Italian Literacy Tutor Interactive Books Video (7yrs) (13yrs)
WELCOME to INTERSPEECH 2011 Firenze August 2011 MANY THANKS! and