Section: Contracts and Grants with Industry

European and National Projects

IST 506909 CHIL: Computers in the Human Interaction Loop

European Commission project IST 506909 (Framework VI - Call 1)

Strategic Objective: Multi-modal Interaction

Start Date: 1 January 2004.

Completion Date: 31 September 2007.

CHIL is an Integrated Project in the new Framework VI programme.

Participants

• Fraunhofer Institut fuer Informations- und Datenverabeitung, Karlsruhe, Germany

• Universitaet Karlsruhe (TH), Interactive Systems Laboratories, Germany

• Daimler Chrysler AG, Stuttgart, Germany

• ELDA, Paris, France

• IBM Czech Republic, Prague, Czech Republic

• Research and Education Society in Information Systems, Athens, Greece

• Insitut National Polytechnique de Grenoble, France

• Insituto Trentino di Cultura, Trento, Italy

• Kungl Tekniska Hogskolan (KTH), Stockholm, Sweden

• Centre National de la Recherche Scientifique, Orsay, France

• Technische Universiteit Eindhoven, Eindhoven, Netherlands

• Universitaet Karlsruhe (TH), IPD, Karlsruhe, Germany

• Universitat Politecnica de Catalunya, Barcelona, Spain

• Stanford University, Stanford, USA

• Carnegie Mellon University, Pittsburgh, USA

The theme of project IP CHIL is to put Computers in the loop of humans interacting with humans. To achieve this goal of Computers in the Human Interaction Loop (CHIL), the computer must engage and act on perceived human needs and intrude as little as possible with only relevant information or on explicit request. The computer must also learn from its interaction with the environment and people. Finally, the computing devices must allow for a dynamically networked and self-healing hardware and software infrastructure. The CHIL consortium will build prototypical, integrated environments providing:

Perceptually Aware Interfaces: Perceptually aware interfaces can gather all relevant information (speech, faces, people, writing, and emotion) to model and interpret human activity, behaviour, and actions. To achieve this task we need a variety of core technologies that have progressed individually over the years: speech recognition and synthesis, people identification and tracking, computer vision, automatic categorization and retrieval, to name a few. Perceptually aware interfaces differ dramatically from past and present approaches, since the machine now observes human interaction rather than being directly addressed. This requires considerably more robust and integrated perceptual technology, since perspectives, styles and recording conditions are less controlled and less predictable, leading to dramatically higher error rates.

Cognitive Infrastructure: The supporting infrastructure that will allow the perceptual interfaces to provide real services to the uses needs to be dramatically advanced. Cognitive and Social modeling to understand human activities, model human workload, infer and predict human needs has to be included in the agent and middleware technology that supports CHIL. Further, the network infrastructure has to be dynamic and reconfigurable to accommodate the integration of a variety of platforms, components, and sensory systems to collaborate seamlessly and on-demand to satisfy user needs.

Context Aware Computing Devices: CHIL aims to change present desktop computer systems to context aware computing devices that provide services implicitly and autonomously. Devices will be able to utilize the advanced perceptual interfaces developed and the infrastructure in CHIL to free the user and allow him instead of serving the device to be served and supported in the tasks and human-to-human interactions he needs to focus. Further, human centered design, where the artistic value, appeal, and look & feel, become important in taking computing devices and human environments to the next level.

Novel services: The above innovations and advances in perceptual interfaces, cognitive infrastructure and context aware computing devices are integrated and showcased in novel services that aim at radically changing the way humans interact with computers to achieve their tasks in a more productive and less stressful way. These services are based on a thorough understanding of the social setting, the task situation, and the optimal interaction that maximizes human control while minimizing workload. Furthermore, some issues of privacy and security are to be addressed since the change human-computer interaction introduced by CHIL also touches a lot of the ways information in which is shared and communicated.

New measures of Performance: The resulting systems should reduce workload in measurable ways. To achieve these breakthroughs in a number of component technologies, the integrated system and a better understanding of its new use in human spaces are needed. Evaluation must be carried out both, in terms of performance and effectiveness to assess and track progress of each component, and the "end to end" integrated system(s). This will be carried out by an independent infrastructure that would also allow any third party to benchmark its findings against the project results after the end of the project.

ANR Project CASPER: Communication, Activity Analysis and Ambient Assistance for Senior PERsons

Start Date: 1 Jan 2007

Duration: 36 months

The consortium consists of INRIA Rhone Alpes, France Telecom R&D and H2AD.

The CASPER project will develop new technologies to respond directly to the increasingly urgent social problem of care for elderly persons. A complementary target group of persons with cognitive deficiencies, including persons with early forms of Alzheimer disease and brain-injured people will also be addressed.

We propose to create a family of devices that provide monitoring services as well as a sense of "presence" between elderly people or people with cognitive deficiencies on the one hand, their family or friends, volunteer helpers of health care professionals on the other hand. The proposed family of devices will consist of a “mother station” assisted by a number of satellite devices for specialised sensing services. The mother station and satellite sensors form a wireless sensor network that uses machine-learning techniques to collectively construct and maintain a model of daily activity. The devices will use "ambient" communication technologies (lights, sounds, movements, vibration) to interact in a familiar, simple and unobtrusive way with their users, and to communicate a sense of presence between elderly people, their family and health care service providers.

The devices will embed and combine visual, acoustic and tactile sensor modalities for observing the daily patterns of activity. Mother stations will embed a panoramic camera, microphones, tactile sensor and bio-metric sensors to note movements, activities and physiological parameters of its owner. The wireless satellite devices will provide specialised sensing to recognize specific classes of domestic activity. For example they may use acoustic sensing to recognize "water sounds" (e.g. shower, bath, basin etc), kitchen sounds (cooking, washing or manipulating food packages), and living room sounds (telephone, TV, Radio or visitors). Incremental machine learning will be used to refine and adapt pre-learned recognition procedures to ambient sounds.

The system will not record images or sound (unless instructed to), but will use embedded real-time machine perception and pattern recognition to recognize classes of activities and organise an activity log. The mother station will maintain a journal of daily activities organised according to location and time, where location will be in the form of topologically associated spatio-temporal map that devices infer from sensor observations. Activity will be recorded as a series of situations organised using 24 hour and 7 day cycles. The activity log can be used to provide un-intrusive "presence" information for family members, as well as more detailed records for health care professionals. Comparison of current activity to a statistical summary of the activity log can be used to sense unusual situations requiring intervention.

The result of this project will be a new technological approach to assist elderly and brain-injured people in maintaining independent living and social interaction. The overall aim of this project is to demonstrate the required technology, and to understand its user requirements and its social implications.

The project will make key contributions to progress beyond the state of the art. These include:

• A new approach to provision of care, assistance, social interaction and social connectedness to older people living independently, combining traditional monitoring with high-level detection of situations, activities, and activity patterns

• Advances in perceptual technologies, affective computing, ambient system design, and sensor networks that will be generic and relevant beyond the use for healthcare monitoring.

The impact of the proposed technologies goes beyond the problem of independent living, health and well being of the elderly. These technologies can stimulate emergence of new commercial industries in a number of areas. For example, affective interaction is expected to provide a major impact on the way that people interact with information technology. Such technology may mark a rupture point in the evolution of informatics and the start in an exponential growth in new applications across a broad spectrum.