Beyond the Smart Home

Tatsuya Yamazaki Universal City Group, Knowledge Creating Communication Research Center National Institute of Information and Communications Technology yamazaki@nict.go.jp Abstract
Due to progress in wired and wireless home networking, sensor networks, networked appliances, mechanical and control engineering, and computers, we can build smart homes, and many smart home projects are currently proceeding throughout the world. However, we have to be careful not to repeat the same mistake that was made with home automation technologies that were booming in the 1970s. That is, [total?] automation should not be a goal of smart home technologies. I believe the following points are important in construction of smart homes from users viewpoints: development of interface technologies between humans and systems for detection of human intensions, feelings, and situations; improvement of system knowledge; and extension of human activity support outside homes to the scopes of communities, towns, and cities In the 1990s, so-called ubiquitous [1] or pervasive computing technologies arose from progress in wired and wireless home networking, sensor networks, networked appliances, mechanical and control engineering, and computers. Using these developed technologies, researchers have started smart home projects in every corner of the world. Each project has a unique goal, and some interesting discoveries have been made. At this stage, however, we have to be careful not to repeat the same mistake that was made with HA technologies; as Intille advises, Researchers and technologies are more cautious in predicting the future of the home, [2]. He also describes, A survey of ongoing work shows that there is a bias in research toward creating automatic home environments that eliminate the need to think about tasks, such as controlling heating and lighting, going to the grocery store, scheduling home appliances, and cooking. I dare say that we must go beyond smart homes that automate all tasks for us. Toward the goal of going beyond smart homes, the following points should be considered: development of interface technologies between humans and systems for detection of human intensions, feelings, and situations because interfaces should not be unidirectional, but interactive and spiral; improvement of system knowledge, for example, by incorporating Web technologies and making it flexible and adaptable to changing lifestyles; and extension of human activity support outside homes because we live not only in houses but also in communities, towns, and cities. Related works are listed in section 2, and section 3 describes in detail the smart home built in our project. In section 4, the Universal City project we are conducting is introduced as an example of smart home extension.

1. Introduction
We can trace the roots of smart homes back to 1970s home automation (HA) technologies. Research has shown that, except for infrared remote controllers of consumer appliances, these technologies failed to improve our lifestyles significantly for several reasons. First, determining economic benefits of HA technologies was difficult. Namely, effects brought about by these technologies were small in comparison with their introduction costs, and they did not contribute to total household energy savings. Also, systems based on HA technologies were so inflexible that new network infrastructures had to be constructed, and these systems were not adaptable to family lifestyle variations. Moreover, HA technologies lacked applications and services that users wanted. Technological capability often lags behind user demand although users sometimes attribute this problem to killer applications and research and development.

2. Related works
Real-life living space test beds are sometimes called smart homes or sensor-embedded houses, and much

2006 International Conference on Hybrid Information Technology (ICHIT'06) 0-7695-2674-8/06 $20.00 2006

research has previously been reported on them. Research on smart homes built within the last decade is reviewed in this section. In 1995, Welfare Techno-Houses were constructed in Japan [3]. The concepts of these experimental houses were to promote independence for elderly and disabled people and improve their quality of life. The Aware Home [4] project is noteworthy among smart home research. In that project, researchers built a three-story, 5040-square-foot home that functioned as a living laboratory for interdisciplinary design, development, and evaluation the future of domestic technologies. The University of Texas at Arlington has conducted the MavHome project since the beginning of the twenty-first century [5]. The MavHome is a home environment that detects home environmental states through sensors and intelligently acts upon the environment through controllers. At the Massachusetts Institute of Technology, the House_n group [6] is working toward a vision where computer technology is ever-present, but in a subtler way than is often advocated in popular culture and engineering paper motivation sections. The group wants sensor-driven pervasive technologies to empower people with information that helps them make decisions without stripping them of their sense of control over their environments. Extended from Robotic Room 2 [7], the Sensing Room was constructed at the University of Tokyo [8].

at the University of Tokyo. In the EasyLiving project, researchers developed prototype architectures and technologies for building intelligent environments. The University of Florida's Mobile and Pervasive Computing Laboratory is developing programmable pervasive spaces in which a smart space exists as both a runtime environment and a software library [10]. The University of Sherbrooke in Canada laboratory is situated at the University of Sherbrooke, the atmosphere there is different from a real-life one. Active Home is located at Information and Communications University of Korea where several experiments of context-aware services based on human behaviors are carried out [12]. UbiHome is another smart space in Korea, where automated light and monitor control based on users situations are demonstrated, for example, in [13]

3. Ubiquitous Home
3.1. Outline
We also constructed a real-life test bed, called the "Ubiquitous Home," for home context-aware service experiments [14]. It is an information and communications technology (ICT) housing test facility for the creation of useful new home services that will be made possible by linking devices, sensors, and appliances across data networks. The layout of the Ubiquitous Home is shown in

Figure 1. Ubiquitous Home layout Although details of daily human behaviors over the long term can be measured in the Sensing Room, there are limitations to imitating real-life behaviors when people are confined to only a small room. The sensing room of the EasyLiving project at Microsoft Research [9] is similar to the Sensing Room Fig. 1. In the Ubiquitous Home, experimenters can collect real-life data as if living in their own houses, not in a laboratory. The Ubiquitous Home differs from other test beds in three aspects. First, we have enhanced sensor ubiquity in it; that is, we have installed many cameras and microphones in each room

2006 International Conference on Hybrid Information Technology (ICHIT'06) 0-7695-2674-8/06 $20.00 2006

and various sensors to monitor locations in every space of the home. We have also set up a remote Japanesestyle room in which we can test extended family connectivity and create a test bed for specific Japanese services. Furthermore, we have introduced robots in home services (Fig. 2).

RFID tag passes through an entrance, an antenna reads information on the tag.

Figure 2. Ubiquitous Home robot The Ubiquitous Home is an apartment comprised of a living room, dining room/kitchen study, bedroom, washroom, and bathroom. In addition to the apartment, a Japanese-style room is provided as a living space for extended family members, such as a grandmother and/or grandfather. Between the apartment and the Japanese-style room, is a computer room called the Network Operating Center (NOC). Above the ceiling is a space where researchers can work. In the space, there are three corridors called "catwalks." Researchers use them for moving around and installing machines and cables. Moreover, there is a 40-cm space between the foundation and floor of the Ubiquitous Home to accommodate free access. The Ubiquitous Home is equipped with various sensors to monitor human activities. Each room has cameras and microphones (Fig. 3) in the ceiling to gather audiovisual information. Respecting residential privacy in such ubiquitous camera and microphone environments is an important issue to consider. Floor pressure sensors (Fig. 4) installed throughout the flooring contain 18-by-18-cm binary detection units and are used to track residents movement or detect furniture positions. Infrared sensors (Fig. 5) installed above the entrance to each room, at foot positions in the kitchen and along the corridor are used to detect residents movement. Two radio frequency identification (RFID) systems are installed in the Ubiquitous Home. One is active and the other passive. The former uses 315-MHz waves, the latter the 2.45-GHz band. Active system scanners, located above the ceiling of each room, detect RFID tags when a person wearing them enters the room. Passive system antennas are embedded inside walls around each room entrance. When a person wearing an

Figure 3. Camera and microphone in ceiling

Figure 4. Floor pressure sensor

Figure 5. Infrared sensor Four accelerometers or vibration sensors are attached to the underside of the bedroom floor in all four corners. Although the role of accelerometers is similar to that of floor pressure sensors, they are more promising for detection of human behaviors because of their higher precision than floor pressure sensors.

2006 International Conference on Hybrid Information Technology (ICHIT'06) 0-7695-2674-8/06 $20.00 2006

Plasma panel and liquid crystal displays (Fig. 6) and speakers are installed throughout the Ubiquitous Home to provide residents with audiovisual contents.

3.2. Context-aware services implemented in the Ubiquitous Home

To provide a service to users, we should consider their contexts. Although the definition of user context may change based on the situation or service, who, where, and when are usually the main contexts. In the Ubiquitous Home, personal identification is obtained from an active RFID tag worn by the resident or face recognition by a robot camera. The residents location is also detected from the active RFID tag, and time is provided by a computer clock that is adjusted by a Network Time Protocol (NTP) server. With the above contexts the following contextaware services are implemented in the Ubiquitous Home. (1) TV program recommendation When a resident orders the robot to turn on the television in the living room, it does so using network commands and selects TV programs that the resident is likely to watch by comparing the Electronic Program Guide (EPG) and the residents TV viewing history. For example, if a word in a TV program in the EPG appears in the viewing history, that TV program is recommended. (2) Cooking recipe display When a resident says a food-related word, the robot selects a recipe related to that word. The resident can continue searching for recipes by talking with the robot until the desired recipe is found, upon which, it is displayed on the display panel in the living room or the kitchen. (3) Forgotten-property check service An RFID tag is attached to each article a resident removes from the Ubiquitous Home. In addition, articles he/she should bring with him/her are listed for each outside destination. When he/she leaves, he/she can verify which articles to take with him/her at the entrance hall using the RFID tag reader installed in the shoebox. We have also implemented service that demonstrates collaboration of a refrigerator, a plasma display panel, and the area RFID system. The refrigerator has a camera inside to capture images of its contents, and the images are displayed on the display panel closest to the resident.

Figure 6. Plasma display on wall The Ubiquitous Home is a highly networked and sensor-embedded home and is considered as the unconscious-type robot that is a robot of which residents are not consciously aware. It autonomously controls appliances on the home network based on sensor information. The robot shown in Fig. 2, on the other hand, serves as an intermediary between the inhabitants and the unconscious-type robot by providing a human-machine interface. This is the visible-type robot that is a robot of which residents are aware and on which they rely.

4. From the Ubiquitous Home to the Universal City

2006 International Conference on Hybrid Information Technology (ICHIT'06) 0-7695-2674-8/06 $20.00 2006

As described in the introduction, users activities cannot be limited to inside their homes. We have to construct smart spaces that connect insides and outsides of homes seamlessly. For this purpose, we have started the Universal City project to build a nextgeneration intellectual living environment hospitable to everyone, including the elderly, people with disabilities, and children. In other words, we are conducting research and development for constructing a society in which an entire town will keep a gentle eye on residents thereof and visitors thereto, and support itself in a variety of aspects, such as outdoor and community activities and lifelong learning. Based on the accumulated research and development results on i) home network infrastructure technologies utilizing a real living-environment-type home ubiquitous environment test bed such as the Ubiquitous Home, ii) technologies for providing situation-adaptive-type services, iii) barrier-free communications technologies as exemplified by intelligent mobile terminals and barrier-free information interfaces technologies, we will address challenges of constructing the Universal City during the next five years. To this end, useradaptation technologies and community-adaptive communications technologies will comprise the pillars of the plan. The key technologies necessary for building the Universal City are depicted in Fig. 7.

Networked robots

Service content

Universal City
Universal interface

Ubiquitous Home
At anytime

Sensor

models to establish technologies for extrapolating user intentions from the data. From the environmental side, the challenges are to extract useful knowledge from information acquired by networked sensors, specify people who need such information, and transmit it to them. Even if we are not transmitting it in real-time, when considering necessary frameworks for building the Universal City, it is vital to enable continuous and precise transmission by storing useful real-world information with detailed information on time and location coordinates at distinct corners of every town. The Internet provides indiscriminant access to information from anywhere in the world as technologies mutually linking entities scattered around the world. Community-adaptive communications technologies are communications infrastructure technologies necessary for building "Ubiquitous and Universal Towns." I believe such technologies will adapt to physical sizes, including homes, communities linking them, and towns containing other facilities and enable flexible information delivery suitable for their content within and/or between entities. Furthermore, since research and development on communications technologies is being conducted in a multifaceted manner, technologies being used differ at each location. To make seamless information transmission/reception possible between heterogeneous communications, including wireless and wired communications, power line communications (PLC), visible light communications, and sensor networks, we must conduct research and development on technologies for comprehensively using those communications.

From and to anywhere

Every product

Everyone

5. Conclusion
I addressed issues that technologists and researchers in smart home research and development should consider. The goal must not be to construct automatic home environments. We must go beyond smart homes that automate all tasks for us. Toward the goal of beyond smart homes, I stated some important points that need to be addressed: development of interface technologies between humans and systems for detection of human intensions, feelings, and situations; improvement of system knowledge; and extension of human activity support outside homes to the scopes of communities, towns, and cities. Although we have not been able to achieve this goal yet, I presented the Ubiquitous Home project that we have previously conducted and the Universal City

Figure 7. Key technologies in Universal City Information and communications technologies will be used further in a variety of situations at home and in offices. We will now refer to people who use systems and receive information and services as "users." Preferences, abilities, and situations vary for individual users at various moments. Where referring to all these factors as a user environment, we have to develop universal designs adaptable to each user environment. To achieve user-adaptation technologies, we must "know" real users. To do so, it is vital to observe real life human activities and behaviors and model human activities and cognizance based on data obtained through such observations. In addition, we will conduct studies on user-adaptation technologies on a range of accumulated and analyzed data, as well as on

2006 International Conference on Hybrid Information Technology (ICHIT'06) 0-7695-2674-8/06 $20.00 2006

project that we will move on to during the next five years.

References
[1] Mark Weiser, The Computer for the Twenty-First Century, Scientific American, pp. 94104, September 1991. [2] Stephen S. Intille, The Goal: Smart People, Not Smart Homes, C. Nugent, J. C. Augusto (Eds.), Smart Homes and Beyond, 4th International Conference On Smart homes and health Telematics (ICOST2006), Assistive Technology Research Series 19, pp. 36, IOS Press, June 2006. [3] T. Tamura, T. Togawa, M. Ogawa, and M. Yoda, Fully automated health monitoring system in the home, Med. Eng. Physics, 20, pp. 573579, 1998. [4] Cory D. Kidd, Robert J. Orr, Gregory D. Abowd, Christopher G. Atkeson, Irfan A. Essa, Blair MacIntyre, Elizabeth Mynatt, Thad E. Starner, and Wendy Newstetter, The Aware Home: A Living Laboratory for Ubiquitous Computing Research, Proc. of the Second International Workshop on Cooperative Buildings - CoBuild'99, 1999. [5] S. K. Das, D. J. Cook, A. Bhattacharya, E. O. Heierman, III, and T.-Y. Lin, The Role of Prediction Algorithms on the MavHome Smart Home Architectures, IEEE Wireless Communications (Special Issue on Smart Homes), Vol. 9, No. 6, pp. 7784, Dec. 2002. [6] Stephen S. Intille, Designing a home of the future, IEEE Pervasive Computing, Vol. AprilJune, pp. 8086, 2002. [7] Taketoshi Mori, Katsutoshi Asaki, Hiroshi Noguchi, and Tomomasa Sato, Accumulation and Summarization of Human Daily Action Data in One-room-type Sensing System, Proc. IROS (IEEE/RSJ International Conference on Intelligent Robots and Systems) 2001, pp. 23492354, 2001.

[8] Taketoshi Mori, Hiroshi Noguchi, Aritoki Takada, and Tomomasa Sato, Sensing Room: Distributed Sensor Environment for Measurement of Human Daily Behavior, Proc. of First International Workshop on Networked Sensing Systems (INSS2004), pp. 4043, 2004. [9] Brumitt, B., Meyers, B., Krumm, J., Kern, A., and Shafer, S. EasyLiving: Technologies for Intelligent Environments, Handheld and Ubiquitous Computing 2000, in LNCS 1927, Springer-Verlag, Sept. 2000. [10] A. Helal, W. Mann, H. Elzabadani, J. King, Y. Kaddourah, and E. Jansen, Gator Tech Smart House: A Programmable Pervasive Space, IEEE Computer magazine, pp. 6474, March 2005. [11] Pigot, H., Lefebvre, B., Meunier, J. G., Kerherv, B., Mayers, A., and Giroux S., The role of intelligent habitats in upholding elders in residence, 5th International Conference on Simulations in Biomedicine, Slovenia, April 2003. [12] Jae-Seon Lee, Kyoung-Shin Park, and Min-Soo Hahn, WindowActive: An Interactive House Window On Demand, 1st Korea-Japan Joint Workshop on Ubiquitous Computing and Networking Systems (UbiCNS 2005), pp. 481484, June 2005. [13] Yoosoo Oh and Woontack Woo, A unified Application Service Model for ubiHome by Exploiting Intelligent Context-Awareness, Proc. of Second Intern. Symp. on Ubiquitous Computing Systems (UCS2004), pp. 117122, Tokyo, 2004. [14] Tatsuya Yamazaki, Human Action Detection and Context-aware Service Implementation in a Real-life Living Space Test Bed, Proc. TridentCom 2006 (Second International Conference on Testbeds and Research Infrastructures for the DEvelopment of NeTworks and COMmunities), March 2006.

2006 International Conference on Hybrid Information Technology (ICHIT'06) 0-7695-2674-8/06 $20.00 2006