Other Links:

Pervasive/Ubiquitous Computing aims to have computing abilities embedded in the world around us, which will provide us an Human Centered Environment. We've been examining the enabling technologies, especially HCI side of pervasive computing. Research topics in our group include:

  • Human Computer Interaction: UI Efficiency Model and Optimization, Natural UIs Innovation on mobile, very large tabletop, display haptics, Web Page Adaptation, etc.

  • Smart Space: Interoperation among devices, Context Aware Software Infrastructure, Multiple Users Multimodal Interaction Mechanism, Indoor Locating, etc.

  • Multimedia Communication: QoS oriented application layer Communication Strategies for group communication and multimedia delivery.

Some topics:

Modeling and Optimization of GUI Efficiency

Making applications more efficient to use is an important topic of research in HCI, especially for the classical GUI/WIMP UIs that are still widely used today. The input efficiency of WIMP user interface can be estimated as the time cost (T) to complete a task according to Fitts' Law where pi and p(i|j) represent the frequency and conditional frequency of widgeti in usage trace. T can be minimized by compressing the length of command sequence or optimizing the layout of widgets.

Optimization strategies are Command Compression and Widget Layout Optimization.

  • Command Compression: Re-constructing command set from usage trace that enables the user task to be completed with minimal number of target acquisition operations.
  • Solution: Entropy-based metric for evaluating input efficiency of a WIMP interface with respect to the statistical feature of usage trace.
  • Advantage:
    1. It produces optimal command set to save input operations in theory.
    2. It requires no semantic information from application and can be used as a general solution.
  • Widget Layout Optimization:Achieving flexible re-layout strategy with minimal layout instability.
  • Adaptive Satellite Cursor associates every potential target with a separate cursor within its vicinity. Pointing a target is done with its "satellite" cursor.
  • Advantage:
    1. Layout of widgets is visually unchanged.
    2. Learning costs associated with the adaptive user interface is largely cut down.
    3. It allows predictive algorithms to be fully exploited, without having to keep their conceptual model simple or to keep the rate of adaptation slow enough not to disorient users.

Researches on UI Efficiency have been Supported by 863 high tech development plan. Papers were published on ACM UIST, CHI, Journal of Software (Chinese), Int. Journal of Human Computer Studies, etc.

Natural Interaction on Mobiles

As pocket devices, mobile phones are always with us, with at least a global wireless infrastructure to support them. They have an increasing number and variety of sensors mobile phones are not only becoming increasingly powerful but also more sensible. Phone Sensor-based interaction gradually draws more attentions from both academia and industry, spreading into lots of topics, like natural user interface, gesture interaction, perceptual interface, tangible user interface, context-aware and activity- based computing.
Early in CHI 2006, we showed Direct Pointer which had the close-loop visual feedback updating the rendered cursor's position on the display with a hand held camera. Direct Pointer was designed to improve our previous work on laser pointer detection, and we pointed the possibility of using phone built-in camera to sense the hand movement. uCam algorithm was then designed to detect hand movement parameters in real time on mobile phone. Some uCam applications like Picture Viewer on phone and Surprise Grabber game demoes the ability of capturing the 3D Motion parameters.

User Activity Recognition on mobile device will benefit personalized real activity based applications. In our method, raw data from multiple sensors is fused into feature vectors to improve accuracy. Reusable atom actions are then clustered. Finally, human activity will be recognized by matching the action sequence. And, the on-body position of the device will be detected in advance to improve the adaptability of the recognition.

Researches on Natural Interaction on Mobiles have been Supported by 863 high tech development plan, NOKIA, MSRA, SIEMENS. Papers were published on ACM CHI, ACM Multimedia, ACM VRST and CSCW.

Large Display Interaction

Displays of large scale which can provide visual output channels and on-surface direct manipulation possibilities, are ideal settings for co-located collaborative activities. Issues lay on large tabletop system, multi-user sharable interaction, and innovation applications.
uTable: a very large-scale tabletop platform, is seamlessly tiled, high resolution with rear projection and multi-touch supported. uTable affords multiple people sitting around, discussing face to face, and interacting at any area of the surface. When constructing uTable, we make great effect to achieve a trade off between perfect visual effect and accurate interaction tracking. The first version of uTable (supporting pen and touch input) was established in 2006. uPlatform is a customizable multi-user windowing system (UI Management) we specifically created for interactive tabletop. Its driving goal is to facilitate the design of innovative user interfaces and window management policies in the context of multi-modal input, multi-task UI and multi-user usage, and at the same time be simple enough for average researchers to quickly learn and use. The application scenarios include emergency command, conference, house design, media sharing, meal ordering, etc, which demonstrate the possibilities of multi-user co-located collaboration on interactive tabletop.

Researches on Large Display Interaction have been Supported by 863 high tech development plan. Papers were published on ACM CHI, UIST, ITS and AVI.

Smart Spaces

Smart Spaces are smart environments integrated with Ubicomp and HCI technologies, allowing people to perform tasks efficiently by offering unprecedented levels of access to information and assistance from the cyber world. Demonstrative projects in PI are Smart Classroom (2000-), SEMIC (2004-), and Smart Home (2009-).

Smart Classroom integrates voice-recognition, computer-vision, and other technologies to provide a real classroom similar experience with natural interfaces. The system is running on the supporting software of Smart Space we built named Smart Platform.
Key Techs.: Natural HCI, Context Awareness, Multimedia Multicasting

SEMIC(Smart Environnent for Multi-user Interactive Collaboration) is designed to help people who work in emergency management centers (ERC) interact with supporting computer systems and collaborate with local and remote people more efficiently. It addresses the spontaneous collaborations among multiple dynamic devices, as well as the impromptu piecemeal interactions between these devices and the people.
Key Techs.:Co-located HCI, Indoor Locating, Service Management including Service Discovery, Service Composition, Service Arbitration, Resource Management.

Smart Home is a smart living environment that we embed networked smart appliances as well as integrate location detection, context-awareness and privacy protection services into people's home with multimodality-based UI to enhance their living experiences. It is developed based on the designed OSGi-based Smart Home Portal called uCore. Key Techs.: RFID-based location detection, Universal Monitor and Controller on phone, OSGi-framework with R-OSGi extension, UPnP, IGRS.

Researches on Smart Space have been supported by 863, NSFC, SIEMENS, INTEL, IBM and Ministry of Education. Papers were published on IEEE Pervasive Computing, IEEE TKDE, TPDS, UbiComp, etc.


ESLWriter is a corpus-based web application geared toward helping those English as a Second Language (ESL) researchers with academic writing in multiple approaches. A well-defined query language and a user-friendly query interface enable users to launch complicated semantic queries into a domain-specific corpus. A collocation checker can automatically examine user's input text and provide writing suggestions when potential errors are detected.
See http://pi.cs.tsinghua.edu.cn/eslwriter