Intelligent space is of a harmonious human-computer interaction principles and techniques of the typical environment, smart space application can also be directly reflected in their specific purposes, such as smart meeting rooms, operations control room, smart classrooms, smart people can take care of home, etc. This article will highlight smart consultation rooms.
Smart Space (Smart Space) is embedded computing, information equipment and multi-modal sensing devices work or living space, with natural and convenient interactive interface to support the people easy access to computer systems services. Smart people working and living space, the process is to use a computer system process, but also human and computer system continuous process of interaction. In this process, the computer is no longer just a passive executor of the operation command explicit information processing tools, but the collaborative task helper who is one of the partners, both interactive collaborative relationship with the harmony. The harmony of this interaction is mainly embodied in people learning to use computer systems and operations will effectively reduce the burden of the interactive entirely spontaneous act of a people. Spontaneous (spontaneous) mean non-binding, non-mandatory and not learning, spontaneous interaction is that the people can the natural first-class data (such as language, gestures and writing, etc.) to interact with the computer system.
Currently, the Pervasive Computing (Pervasive Com-puting) is a technology research and application of the hot spots, and spontaneous interaction is ubiquitous computing from desktop computing interactive mode binding of the key issues of great research value, as of harmonious human-computer intelligent space interaction principles and techniques of the typical environment.
Understanding of intelligent space
Pervasive Computing will enable computing and information services to suit the way people use common in our surroundings, in the past isolated information space and physical space to each other together. In this fusion of space, people can at any time, transparent access to computer systems services. Pervasive Computing in the information space and physical space of the integration can be reflected in the different scales, its in the room, the building of this scale is the embodiment of intelligent space.
NIST (National Institute of Technical Standards) given smart space with the functions and services provided to users include:
● to identify and perceive the user as well as their action and purpose, understand and predict the course of the user needs to complete the task;
● Users can easily with a variety of information sources (including equipment and data) to interact;
● users carry mobile devices seamlessly with the infrastructure of smart spaces to interact;
● provide a wealth of information display;
● provide intelligent space in the experience of records to retrieve at a later playback;
● support the collaborative work space as well as many remote users immersive collaboration.
Some scholars have pointed out that the future is unlikely that there is a global integrated pervasive computing system, there will be many because management divisions, geographical boundaries and cultural divisions separated from each other, there are clear boundaries smart space systems. These intelligent systems can have certain spatial interaction, and a variety of portable devices can transparently move between them.
Characteristics of intelligent space
Current international research on smart space to carry out quite extensive, that the smart space research in ubiquitous computing important role. These research programs, intelligence space to show them a different application form, and was given the researchers hope that each of the characteristics of pervasive computing, the analysis of these research programs can analyze the three main characteristics of intelligent space, and three stages.
First of all, in a computing environment, computing devices are no longer as in the desktop computing model, requires the user sitting in front of the computer or in a fixed position to complete computing tasks. A large number of computing devices, multi-modal interactive technology module, scene perception (context awareness) module is embedded and hidden in the actual physical environment, these modules need to be able to collaborate with each other and can take the initiative to provide services for the users to make smart space to have three-dimensional continuous interaction channel. For example, a smart study can detect the user to read books in which, it may open the curtains to provide users with adequate lighting; as time goes on, night fell, intelligent study may also be based on current lighting conditions, the gradual strengthening of the room lighting.
Secondly, an important feature of intelligent space called nomadic services (cyber foraging). It is carried into space the user's wireless handheld devices can take full advantage of its surrounding infrastructure, the relatively strong capacity of the equipment, while its modules can run in the other modules and space to interact and collaborate to jointly as provide enhanced services. This is the information space and physical space, an embodiment of integration. This integration makes the calculation of distance on the meaning and the exact opposite of traditional distributed computing. Traditional distributed computing is the attempt to destroy the network distance, while the smart space is emphasized from the user to find the latest resources and services. In turn, the infrastructure should be able to use mobile devices carried by the user functionality to provide users with personalized or enhanced service. For example, when user A in a smart conference room, the system checks to B to A made an important E-mail, and hope that a soon be able to read its contents. Smart meeting room they should be based on the current scene (the meeting), and carried by a mobile device (cell phone or PDA) to judge, forward this E-mail to the phone, and through vibration to remind the A, rather than through the conference room The large screen is displayed. If only a single stay smart meeting room, the E-mail may be by voice prompts, displayed on the big screen.
About the author: SHI Yuanchun
Department of Computer Science Tsinghua University professor, deputy director of Department of Academic Committee, Human Computer Interaction and Media Integration Research Institute, part-time member of China Computer Society, the Technical Commission of Pervasive Computing, IEEE Beijing Section Education Committee. MIT Artificial Intelligence Laboratory in for a senior visiting scholar. Research interests include human-computer interaction, distributed multimedia information processing, pervasive computing, distance education.
Finally, the difference between the space should be able to interact spontaneously occur. When a space resources can not meet the needs of users, the smart space should be based on the user's request to close the space for the user making the request to complete the task. For example, in a smart space users need to print a color image, because the space is only black and white printers, smart space may deny the user's request and telling the user can not complete the task. However, if the smart space environment found in the vicinity there is a color printer, it might prompt the user with the location of the color printer, and asks the user whether to use the printer to print.
Stage of development of smart spaces
The three points enumerated above can be seen as the relationship between the three has a progressive characteristic. In fact, we believe that these three characteristics is the development of intelligent space characteristic of the three different stages. Figure 1 describes the development of intelligent space the three phases, and explains each stage of key concern and research.
Independent Intelligent Space (Individual Smart Space) system is a smart first phase of space research. At this stage, the main research content of the module space of communication and coordination, natural human-computer interaction interface, but also requires a system with persistence and transparency features. To support coordination between modules, one of the coordination model, multi-Agent systems and communication language, to build intelligent software support platform space, and apply a lot of touch interfaces, sensors and sensing modules to be more friendly human-computer interaction. Loosely coupled application systems to improve the durability of the system, and embedded technology enables computing devices to retreat to the environment, and disappear from view.
Smart Space Development is open to the second stage of development of smart spaces (Open Smart Space), while concerned about the large number of applications to mobile devices. Mobile devices as users roam in space, they can discover the existence of computing environment, while spontaneous interaction with the computing environment. This phase of research involves the discovery of a computing environment, heterogeneous communication channel of exchange, resource management, agency and technology of composite services.
In fact, we believe that in the future, intelligent space structure may constitute a level. Between space and space will be connected, and a single smart space may together constitute a large space (such as Tsinghua University in the media all the rooms constitute the entire Tsinghua smart media). The space also may continue to pose a more complex space (for example, the media Tsinghua University, Tsinghua University network, software Tsinghua University, Tsinghua performance by Tsinghua and other electronic systems constitute the main building of the Tsinghua University of the East). Eventually, the smart space will constitute a series of intelligence community (Smart Community). Intelligence community in the form of intelligent space will be how the organization together, how to interact across space and how to access resources across space is the need for research.
Be noted that these issues are in accordance with the logic instead of time-relations of (as shown in Figure 1, left to right in the development, the Xin's problems continue to join). Connection between the various stages using a modulation symbol to indicate the original solution to the problem in the process of accession by new problems that make them more complicated. Stages of the view also that the issues raised the previous stage does not mean they are not important at a later stage, but we do not focus on the latter stages of consideration of these issues, but rather assume that these problems have had a better solution. Our focus in the previous stage on the basis of research carried out at a later stage.
Research on the current terms, most projects have been on the first phase (coordination mechanism within the smart space) for a more in-depth study, research began to focus on the second phase of the transfer. A small number of items (such as MIT's AIRE program) has already begun on the third stage of the issues a certain amount of research.
Prototype intelligent consultation room
In the information age, when faced with major emergencies, decision-making groups need in a very short period of time to understand large quantities of complex information and decision-making personnel in full of the exchanges and discussions between to quickly Zuochuzhengque the decision-making.
Decision-making process is one of the information integration process, the simple role of decision support software system is limited, in particular the use of man-machine barrier method is not conducive to policy-makers to access and exchange information. Only decision-makers and information system has been integrated together between the two natural and efficient interaction channel to make full use of information, efficient decision-making.
Intelligent consultation room (SEMIC: Smart Environment for Multi-user Interactive Cooperation) to major emergencies for the application of strategic decision-making background, through the harmonious development and integration of advanced interactive technology to provide policy makers with an efficient access to information, communication working space, thus contributing to significantly improve the efficiency of decision-making.
Consultation room in smart, relevant information from multiple sources will be displayed in the consultation room integrated three-dimensional physical space, in consultation with staff on multiple display surface, natural and easy way to interact directly with information systems, or remote full collaboration with colleagues in the exchange, the entire consultation process will also be automatically stored as an index of decision-making records available, or reference case. Intelligent consultation room is a typical multi-user environment harmonious human-computer interaction, aim to decision makers in complex cases, as soon as possible to understand and grasp the current situation and quickly make informed decisions, the key is to open information integration , to provide multi-user support natural interaction technology.
Figure 2 is a smart consultation room scene photos of a prototype system. The prototype system based on a 5 m × 6 m room, development and integration of hardware devices are: an interactive display wall, a large touch screen display, an interactive desktop, a Tablet PC, 2 PDA, and more features interactive pen (with a per user), linear microphone array 2, a set of indoor positioning system (including seven receivers and five user badges), the process used to identify interactions pen and record 4 cameras , 7 sets of the host computer. Development and integration of software modules are: Multi Agent-based software support platform, the process of recording and retrieval, seamless data management, remote collaboration, handwriting recognition and labeling GIS geographic information system.
Figure 2 Experimental system of Intelligent consultation room
Consultation room smart intelligent pervasive computing based on space technology to create a comprehensive interactive information display and harmonious environment, will greatly enhance the decision-making speed and reliability. Its innovative feature is mainly reflected in:
● a new natural and easy way interaction, decision-makers can focus on information itself, will substantially increase the efficiency of decision-making;
● loosely coupled computing platforms can better integrate with existing information systems quickly as possible to enhance the coordination of multiple systems;
● remote real-time depth of collaborative decision-making process increased the efficiency of remote collaboration can effectively reduce the risk of decision-making;
● Ultra-media recording and more clues to the process of indexing techniques for decision analysis provides a real comprehensive records.
Integrated information display
Need to understand the process of consultation and exchange of various information is displayed in the room integration, creating a full range of information space. According to decision-making process of information access in different ways, the configuration of different types of display devices (such as large Mianji interactive display wall, interactive desktop, touch Hanging Ban, portable terminals), an information display channel to change the current interaction with the information channel Xiang Separation of application models are supported in the information display surface on the direct manipulation of information content, providing users full and efficient information access interface. Meanwhile, various display devices connected through the underlying computing platform, which kept the information in the migration of different visual display devices easy.
Consultation process with new and important source of information, such as portable notebook or new data from the network, intelligent consultation room by the underlying computing platform, the agreement automatically find sources of information, and scheduling software and hardware resources to provide appropriate scheduling and information display, without complex manual configuration.
1. Software support platform - Smart Platform
Software support platform is in consultation room smart computing platform system software, consultation rooms for the various hardware and software intelligence coordination and communications module, all modules will eventually form a coherent whole. The main problems facing the platform to integrate with a variety of heterogeneous computing, sensing capability of interactive devices, the interaction of data between devices, resource discovery, collaboration.
Smart Platform software platform based on multi-Agent Model, because: it encourages the behavior of autonomous computing entities, the composition and structure of the system to adapt to the dynamic nature; Agent package level and higher level of abstraction can reduce the difficulty of building complex systems; calculated by inter-entity Agent higher level coordination between the languages, coupling is small, robust and beneficial to the system extensions.
Figure 3 is the design model based on multi-Agent system software architecture, (a) that the underlying operating system software and network at the top of the middleware, applications, resources and services will be packaged as Agent, the system's communication layer , coordination and common services layer will provide the Agent of the operating environment and application development tools for the upper; (b) that the system run-time structure: Container Management stand-alone local Agent, DS (Directory Service) Agent responsible for global registration, management and operating environment message forwarding. Its characteristics are:
Fig 3 Development Platform for Smart Platform Agent
● loosely coupled structure of the system has good scalability, and help to improve system robustness.
● point to point communication channel and communication channel level forward combination of both the Agent information and real-time data flow between the need for interaction.
● Coordination of layer-based news groups publish / subscribe model, subscribers can have a flexible size controlled and spontaneous interaction between Agent; XML-based message format described ability to support heterogeneous, easy development and porting.
● sufficient universal service support for the development of application layer functions.
2. Seamless Data Management - Liquid
Smart Platform and seamless coordination layer data management module (Liquid) to the unified management of computing devices are stored in different information sources (in file size). From the users point of view, into the consultation room, the information on the different computing devices are placed in a system, users do not care about upload and download information, simply use a function described below, interactive document can be different conveniently displayed on the display device, switch, tagging information so that users can focus primarily on discussion and information to understand the process, without much regard to the details of computing systems. Remote users are allowed to access after the consultation room can also use the module to provide seamless data management and sharing of information. Figure 4 is a diagram of the module.
Convenient way to interact
In a variety of display device integrated information space, decision makers can be the physical environment (such as walls, desktops), daily utensils (such as pens, laser pen), new information devices (such as PDA, microphone array), and voice commands, etc. natural and easy way to interact with information systems without reliance on the traditional mouse and keyboard, so that unskilled workers on the computer can directly access, process information. Prototype system is mainly through voice commands and more on the pen display surface to provide direct support for interactive technology.
1. Multifunction interactive document - uPen
In the consultation room, including a variety of display devices such a three-dimensional interactive space, with the traditional mouse and keyboard to interact is quite complicated: each display device separately with the mouse or keyboard; for inexperienced computer users, This interactive approach is very inefficient; for some operations, such as drawing icons, even skilled computer users with the mouse and keyboard is too much.
We designed and implemented a multi-functional interactive document uPen, it is a pressure sensor touch pen can be fired laser, pen, who also left and right mouse button laser emission and a total of three function keys. Combination touch pad and computer vision technology, users can use a uPen convenient way to the room in consultation with the various display devices to interact with. This gesture interaction model out of mouse and keyboard, allowing users to various locations in the room can interact with the display device, and interactive and convenient way of reunification.
Each course in the use of a uPen can send only to the system ID information for multi-branch uPen also provide the basis for the work. Combination of indoor positioning systems, they can interact to determine the current state of use of personnel for the system of active service and user related to the process of recording possible.
2. Interactive display wall - iWall
Display wall is a case of people working together effective means of information presentation. At present, the most senior command or decision-making centers have used some display wall technology to provide multiple simultaneous presentation of information. Interactive Intelligence consultation room including the use of projector display wall devoted to the main screen common wall, and the display area composed of several small auxiliary screen (Figure 2, the wall projection and the right of the wall and the bottom assisted small screen). Main screen size and the number of secondary screens can be easily extended as needed.
Users of the interactive display wall to complete the operation by uPen, the basic principle is to use visual technology to track the laser point to issue identification uPen interactive display wall in the location, the user can change the hand-held laser point uPen location and send wireless command signal. System under identification from track and receive the laser point to the wireless command signals to perform operations, such as changing the content of interactive display on the wall to complete the main and auxiliary switching between display.
3. Interactive desktop - iTable
More than discuss and exchange in decision-making process, decision makers still like to sit around the table and use pen and paper methods to communicate, reason is more quick and intuitive operation, but also to facilitate the exchange of people face to face. But lost in this way the computer can provide many of the documents shows that collaborative editing features and networking capabilities. We designed the interactive desktop (iTable) combines the advantages of two ways, namely: the user can use a pen or a finger directly at the large size of the desktop display plane interact with the computer, but also to retain the traditional conference table to discussion.
iTable combined uPen, can provide many solutions simultaneously. Basic principle is based on pressure sensing and visual tracking techniques. The surface equipment interactive desktop camera tracking uPen trajectory, the user was holding uPen pen with pressure sensor, when the tip contacts to the interactive desktop, through uPen track, document the current compression of time and their own ID, the system will be able to distinguish more pens on the desktop at the same time operation of the current position and ID. Further consideration to the user in the process of drawing pixels of continuity and motion characteristics, you can put the pen strokes and their respective counterparts drawn up to address the people on the desktop simultaneously on the issue, currently supports two parallel work.
4. Indoor Positioning System - Cicada
Intelligent consultation room is more than a typical collaborative work environment. Use of indoor positioning system, microphone array, and can distinguish between staff interaction identity information document to automatically distinguish between policy makers and their current state of activities. On this basis, combined with the specific decision-making process for policy makers to match their active duty service, to improve decision-making efficiency.
Meanwhile, the intelligence staff in consultation with the indoor and remote decision-makers can be different from the traditional video conferencing systems for remote collaboration fully. In addition to having intelligent consultation room video and audio channels, for policy makers in different locations to provide a wealth of information sharing and ease of interactivity, so that all parties fully aware of each other's decision-making process based on the idea, in a shared workspace sufficient to overcome the spatial separation of collaboration.
Consultation room is a typical multi-user work environment, support staff work effectively with different identities, an important prerequisite is the ability to distinguish between different staff, testing the activities of the current personnel situation. Tsinghua University developed a new indoor positioning system - Cicada, which is based on radio frequency and ultrasound to measure the distance to the time difference. Cicada not only on the stationary and moving objects accurately positioning (static error is less than 5cm), and has a wide area to the type of work. Because of its small and light (as shown in Figure 6), can be configured in the user's badge and other portable items.
Figure 5 Cicada system architecture
Figure 6 electronic badges and receiver
Decision-making process automatically record
Record of decision-making process is an important reference, a simple written records, or video recording is not enough, the former lost decision-making process is crucial for understanding the data itself, which can restore the information accurate, while the video itself, the lack of structural not easy to retrieve. Intelligent Room can be achieved in consultation with the events of space in the data stream of multi-thread synchronization record, index and playback.
We developed the process of recording the whole process of decision-making module can do multiple cues simultaneously recorded data stream, including the interactive display walls, interactive desktop display various information such as equipment, and data, decision makers interact with equipment operation, such as the main screen interactive display wall switching process, interactive desktop, live action and basic audio, video, etc.; these different streams of data leads in time to keep pace, so playback can be fully reproduce the decision-making process. Recorded with the XML language to describe the data to ensure interoperability. In order to facilitate quick browsing and searching records, the process of recording module from the user, time, location and events to automatically build an index of four dimensions of the structure, and developed a user-friendly graphical query interface, supports fast playback and graphical queries.
Link: Smart Classroom
Pervasive Computing is an important issue is how the application of the study and the current combination of distance education is a good point of integration. Figure 7 Example of a Tsinghua University developed a smart classroom system.
Figure 7 Schematic diagram of smart classroom system
Intelligent Classroom (Smart Classroom) is an intelligent interactive space technology to rely on the true enhanced teaching and learning environment. In addition to traditional classroom podium, desks and chairs, smart classroom podium above the projection of the electronic notes (shown in Figure 7), the left side of the wall shows the situation of remote students, and administrators of the head of virtual classrooms. Operation of the system need to support multiple computers, but the classroom teachers and students do not need to face and operate the computer itself. Traditional classroom teacher can be the same as in the natural and convenient to carry out teaching activities, he had writing on the blackboard, voice command, by way of pointing laser pointer and computer-assisted instruction to facilitate distance learning, distance learning to get rid of restrictions on the desktop interaction models. Meanwhile, the number of students who participate in distance learning in different access modes via the Internet to join the class. The classroom can also automatically determine the focus of the current class, automatically fit the lens of the video sent to remote students. Remote students and the classroom as students participate in classroom teaching, such as speaking and writing on the blackboard and so on. Lecture course can also be automatically recorded into searchable multimedia compound documents, courseware used as a field.
作为一个研究实验平台，智能教室在智能空间和多媒体远程交互等课题方向上已取得了一定进展: 多模态信息融合、智能空间中基于上下文的行为语义、智能交互空间的分布计算环境、内容相关的自适应传输、可伸缩群组交互机制、无缝的可移动性、过程记录、基于内容的多媒体检索等。这个课题还集成了清华大学计算机系多年来在计算机视觉和语音处理方面的优秀成果，包括人的跟踪、基于生物特征的身份鉴别、虚拟人合成等。该课题成果已经投入清华大学继续教育学院的远程教学的实际应用。传统的远程教育系统中的教师是面向计算机的屏幕，而不是学生，这阻碍了教师与学生之间的面对面的交流。在基于智能空间的远程教育系统中教师可面向学生进行教学，同时又可充分地享受信息访问和交互的服务，从而把远程教育技术提高到一个新的水平。
目前，清华大学智能空间的研究已经从面向一个教师为主的教室环境，发展到面向多个用户的会议室环境，随着用户数和开放程度的增加，交互和服务的复杂度将增加维度，面临的问题将更具挑战性。