Home
   Info
   Topics
   Projects
   Contracts
   Publications
   People
   Map
   Italian
print 
SOUND - Role of multimodal feedback in the execution of robot-assisted motor tasks

type: Ateneo
period: 2010-2012
head: prof. Giulio Rosati

description: Interactive robotics is one of the most recent developments of the robotics and mechanical systems control fields, and represents one of the major challenges both in the civil and in the industrial fields. Man-machine interaction can help the humans to manage complex robotic/automatic systems (teleoperation systems, haptic interfaces), allowing to use such systems in the application fields in which task complexity requires the human to be inside the control loop (space robots, robotic surgery systems, underwater robots, etc.). On the other hand, by building machines able to operate in direct contact with the human, the enormous potential of technology can be exploited in very critical application fields such as motor rehabilitation therapy, providing not only the patients with novel instruments and protocols for motor recovery, but also physicians with powerful tools for patient assessments and for therapy standardization. This project proposal aims to investigate the role of multimodal feedback in human-machine interaction, to model the influence of the different forms of feedback (mainly: visual, haptic and acoustic) on the execution of robot-assisted motor tasks. A deep understanding of those mechanisms is fundamental for the definition of new guidelines for the design of interactive robots, able to maximise the engagement of the human in task execution by means of a proper modulation of the sensory information rendered. The key hypothesis behind this work is that motor control is based on sensory information, that is compared to the estimated sensory inputs to update the internal model of the environment and of the limb and to correct the motor command. With this approach, the robotic system is considered as a generator of sensory inputs, whose appropriate combination can lead the human to learn and quickly execute the motor task. The research team has a solid experience in the development of multimodal interfaces and interactive robotic systems (both general-purpose haptic displays and rehabilitation robots), that will be the starting point for this research. A series of experimental tests will be defined, based on the use of both existing prototypes and specifically developed interfaces, with the aim of creating and validating a model of human-robot interaction accounting for human engagement and effort as a function of the major sensory information provided by the robotic system (haptic, visual and acoustic feedback). The results of this project will have a major impact not only in the rehabilitation robotics field, that is the main research field of the principal investigator, but also in the development of novel teleoperation and simulation systems, in which the machine must provide the operator with a virtual guide to let him/her learn and execute a motor task in safe conditions and in a reasonably short time.


partners:
University of California at Irvine, DII


Activities

Auditory feedback in robot-assisted rehabilitation

Publications

D. Zanotto, G. Rosati, S. Spagnol, P. Stegall, and S. K. Agrawal
Effects of complementary auditory feedback in robot-assisted lower extremity motor adaptation
IEEE Transactions on Neural Systems and Rehabilitation Engineering, 2013 (in press)

F. Oscari, R. Secoli, F. Avanzini, G. Rosati, and D. J. Reinkensmeyer
Substituting auditory for visual feedback to adapt to altered dynamic and kinematic environments during reaching
Experimental Brain Research, 221(1):33-41, 2012

S. Spagnol, M. Geronazzo, F. Avanzini, F. Oscari, and G. Rosati
Employing spatial sonification of target motion in tracking exercises
In Proceedings of the 9th Sound and Music Computing Conference SMC 2012, Copenhagen, DK, July 11-14 2012

D. Zanotto, G. Rosati, F. Avanzini, P. Stegall, and S. K. Agrawal
Robot-assisted gait training with complementary auditory feedback: results on short-term motor adaptation
In Proc. of the 4th IEEE RAS/EMBS Int. Conf. on Biomedical Robotics and Biomechatronics BIOROB 2012, pages 1388-1393, Roma, Italy, June 24-27 2012

G. Rosati, F. Oscari, S. Spagnol, F. Avanzini, and S. Masiero
Effect of task-related continuous auditory feedback during learning of tracking motion exercises
Journal of NeuroEngineering and Rehabilitation, 9(79), 2012

G. Rosati, F. Oscari, D. J. Reinkensmeyer, R. Secoli, F. Avanzini, S. Spagnol, and S. Masiero
Improving robotics for neurorehabilitation: enhancing engagement, performance, and learning with auditory feedback
In Proceedings of the IEEE 12th International Conference on Rehabilitation Robotics ICORR2011, pages 341–346, Zurich, CH, June 29-July 1 2011

R. Secoli, M.-H. Milot, G. Rosati, and D. J. Reinkensmeyer
Effect of visual distraction and auditory feedback on patient effort during robot-assisted movement training after stroke
Journal of NeuroEngineering and Rehabilitation, 8(21), 2011

R. Secoli, G. Rosati, and D. J. Reinkensmeyer
Using sound feedback to counteract visual distractor during robot-assisted movement training
In Proceedings of the IEEE 8th International Workshop on Haptic Audio-Visual Environments and Games HAVE2009, Lecco, Italy, November 7-8 2009

Home | Info | Research | People | Map | Italian