WO2006111935A1 - Activity-related document mangement - Google Patents

Activity-related document mangement Download PDF

Info

Publication number
WO2006111935A1
WO2006111935A1 PCT/IB2006/051229 IB2006051229W WO2006111935A1 WO 2006111935 A1 WO2006111935 A1 WO 2006111935A1 IB 2006051229 W IB2006051229 W IB 2006051229W WO 2006111935 A1 WO2006111935 A1 WO 2006111935A1
Authority
WO
WIPO (PCT)
Prior art keywords
document
trigger
context
active
list
Prior art date
Application number
PCT/IB2006/051229
Other languages
French (fr)
Inventor
Markus Gerardus Leonardus Maria Van Doorn
Original Assignee
Koninklijke Philips Electronics, N.V.
U.S. Philips Corporation
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Koninklijke Philips Electronics, N.V., U.S. Philips Corporation filed Critical Koninklijke Philips Electronics, N.V.
Publication of WO2006111935A1 publication Critical patent/WO2006111935A1/en

Links

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/90Details of database functions independent of the retrieved data types
    • G06F16/95Retrieval from the web
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/40Information retrieval; Database structures therefor; File system structures therefor of multimedia data, e.g. slideshows comprising image and additional audio data
    • G06F16/43Querying
    • G06F16/435Filtering based on additional data, e.g. user or group profiles
    • G06F16/436Filtering based on additional data, e.g. user or group profiles using biological or physiological data of a human being, e.g. blood pressure, facial expression, gestures

Definitions

  • the present invention relates to the field of context aware systems, ambient intelligence environment systems, management of documents for managing user activity- related devices, and behavior-specified linking in browsers.
  • a user may be simultaneously using several networked devices, such that it would be desirable for the user's actions or activities with respect to one device or with respect to one activity to affect a second device or second activity.
  • more than one person often shares the environment, which causes activities of one user to affect the environment and thereby impact the second user sharing the environment.
  • the user may activate or control the operation or state of a device other than the device with which the user is directly interfaced or engaged.
  • Such a browser may provide an interface to a user of an active document with a link, the document corresponding to a first user activity, and additional active documents corresponding to other activities of the user and the devices associated with those other activities. That is, each activity of a user could be represented by a document and the ambient intelligence browser can at any time support 0 to N activities (and corresponding documents) as active, N being a positive integer.
  • Each document describes how the presentation is rendered on one or more devices, that is where multimedia elements are to be presented in space (on which device and on which part of the screen, for example) and in time (some multimedia objects begin to play when others have stopped).
  • An example of a document language that supports this is the SMIL language defined by the W3C (http://www.w3.org/AudioVideo/).
  • the document also describes how users can interact with the application by means of forms that describe user interface widgets such as submit buttons, dropdown lists, text fields or the like.
  • An example of a document language that supports this is XForms (http://www.w3.org/MarkUp/Forms/). XForms is intended to be used in conjunction with a web presentation language such as HTML or SMIL.
  • the document has linking functionality.
  • XLink http://www.w3.org/XML/Linking
  • XLink http://www.w3.org/XML/Linking
  • HTML or SMIL a presentation language like HTML or SMIL.
  • XLink http://www.w3.org/XML/Linking
  • XLink defines a "show" attribute that is also used to communicate the desired presentation of the ending resource (end point of the link; the music playing document) on traversal from the starting resource (the begin point of the link; the webcam document).
  • XLink and in presentation languages like HTML and SMIL for which XLink was meant, only one document at any given time is active. In such systems, only one document has focus.
  • WO Patent 2003/101045 is directed to a physical hypertext language, a way of describing how real world devices can be controlled by a declarative language.
  • WO Patent 2003/101045 does not describe a trigger language module that could be used in a hypertext language that describes the interaction of the user with devices in the physical world or monitoring conditions or pre-conditions associated with such triggers.
  • U.S. Patent Application Publication No. 2003/0182474 is directed to a way of mapping high-level device controls on low-level device implementations in a generic way, including receiving a high-level device control command, generating a specific device control script based on a device interconnect model and the high-level device control command received, interpreting the generic device control script, and transmitting specific device control commands to a device through a device interface.
  • This reference does not teach that a user without conscious effort or even awareness can set or retract trigger modules for applications.
  • U.S. Patent 5,611,050 is directed to a system that grants interaction requests that have locational and contextual attributes that are consistent with specified interaction policies and denies those requests that are inconsistent.
  • the invention electronically monitors contextual information concerning user and machines, including state and location information including proximity, and traverses links if the conditions for the link are valid.
  • U.S. Patent 5,611 ,050 does not disclose a system in which the system controls or provides the event information for traversing the link without conscious effort by the user of the system.
  • U.S. Patent Application Publication No. 2002/0035404 is directed to a set-top box that controls the operation of certain devices designated in the script in a way somewhat similar to the way described in WO Patent 2003/101045.
  • the patent describes that each of the scripts is initiated based on a corresponding triggering mechanism that can be set by the user via a graphical user interface (GUI).
  • GUI graphical user interface
  • the user enters the trigger and its firing preconditions in the GUI.
  • the system then waits for the preconditions to become true and fires the trigger.
  • U.S. Patent Application Publication No. 2002/0035404 does not disclose a context aware system in which the user does not consciously set/deactivate triggers.
  • a context aware environment management system, method and processor- readable medium may include a document parser configured to parse an active document that specifies setting information for user activity, the active document containing a trigger-behavior variable and a context trigger that includes a link to at least one target document specifying setting information for user activity and preconditions for the link; a context trigger list manager configured to add the context trigger to a list of active context triggers when the trigger-behavior variable indicates that the context trigger is to be added; such that the context trigger list manager configured to monitor the list of active context triggers, and to traverse only the links of context triggers included on the list whose pre-conditions are satisfied.
  • the document parser could be configured to parse a second document specifying setting information for user activity containing the context trigger and a second trigger-behavior variable associated with the context trigger; and the context trigger list manager may be deployed to remove the context trigger from the list when the second trigger-behavior variable indicates that the context trigger is to be removed from the list.
  • the document parser may reach the context trigger in the active document based on an event related to the user activity specified by the document.
  • this system may be an ambient intelligence engine that includes a browser and the link may be a behavior-specified link managed by the browser.
  • the document manager of the ambient intelligence engine may be able simultaneously to maintain at least one additional document as an active document.
  • Figure 1 illustrates a schematic view of a context aware environment management system according to an embodiment of the present invention.
  • Figure 2 illustrates a browser module according to an aspect of the present invention providing user interface with active documents.
  • Figures 3A and 3B illustrate a flowchart for an operation of a context aware environment management system according to an aspect of the present invention.
  • FIG 4 is a flow diagram showing aspects of system states according to an embodiment of the present invention.
  • the following discussion and the foregoing figures describe embodiments of Applicant's invention as best understood presently by the inventor however, it will be appreciated that numerous modifications of the invention are possible and that the invention may be embodied in other forms and practiced in other ways without departing from the spirit of the invention. Further, features of embodiments described may be omitted, combined selectively, or as a whole, with other embodiments, or used to replace features of other embodiments, or parts thereof, without departing from the spirit of the invention.
  • the figures and the detailed description are therefore to be considered as an illustrative explanation of aspects of the invention, but should not be construed to limit the scope of the invention.
  • the context aware environment management system 1-1 includes several modules, which will be described below. Modules of the activity-related document management system 1 -1, or portions thereof, and/or the activity-related document management system as a whole, may be comprised of hardware, software, firmware, or a combination of the foregoing, however some modules may be comprised of hardware for example, while other modules may be comprised of software, firmware or a combination thereof.
  • modules of the activity-related document management system need not all be located or integrated with the same device.
  • a distributed architecture is also contemplated for the activity-related document management system, which may "piggy-back" off of suitable modules provided by existing devices.
  • the database 1 -2 may be embodied on a storage device such as on a hard drive of a personal computer, a personal video recorder, an entertainment system, an electronic organizer, a personal handheld device, a Jaz drive, or may be embodied as a commercial storage facility, such as a disk drive.
  • a commercial storage facility may include a storage space provided by a service provider, or residing in the network.
  • the database 1-2 may include several storage devices that are connected, such that organization or grouping of content items on two or more of such devices is possible.
  • the data may be distributed over devices in a peer-to-peer network.
  • data derived from sensors may thus be handled.
  • the database may be understood to include one or more storage media, such as disks, including CDs, DVDs, zip disks, floppy disks, data cartridges, or the like, which can be loaded onto and retrieved by the database 1-2.
  • the context aware environment management system 1- 1 is also capable of retrieving content via a network 1-9, such as a LAN, WAN, the internet, or the like, and that the database may be remotely connected, such as via a network, including the internet.
  • context aware environment management system 1-1 includes a document manager 1-11 which manages documents currently active in the system.
  • context aware environment management system 1-1 may include a browser or be logically connected to a browser, or browser module, as shown in Figure 2, including an ambient intelligent browser.
  • the ambient intelligence browser may also be understood as comprising a module or network node outside of the context aware environment management system 1-1, for example as a kind of navigation browsing assistant.
  • the context aware environment management system 1-1, including the document manager 1-11 may be understood as the ambient intelligence browser plus the extra functionality discussed herein and shown as the context aware environment management system 1-1.
  • device 1-52 and sensor 1-51 may be connected in a wired or wireless mode directly to the context aware environment management system 1-1 or to a device physically integrated with the context aware environment management system 1-1, or as shown in Fig. 1 may be connected to the context aware environment management system 1-1 via a network 1-9, such as wired home network application, or other type or wired or wireless network, including a LAN, a WAN or the internet.
  • a network 1-9 such as wired home network application, or other type or wired or wireless network, including a LAN, a WAN or the internet.
  • the context aware environment management system 1-1 may provide for several simultaneously active documents, shown as display document, 2-1, telephone document, 2- 2, and phone ringing document 2-3.
  • Such documents may control devices associated with activities performed by the user, such as CRT monitor or flat panel display or the like controlled by display document 2-1, telephone function controlled by telephone document 2-2, and phone ringing functions controlled by the phone ringing document 2-3.
  • Several sets of such documents may be simultaneously active, each set corresponding, for example, to a different user.
  • the browser module shown in Figure 2 may be integrated with the document manager 1-11 of the context aware environment management system 1-1 or be a separate module, as discussed.
  • a document or a set of several documents may be created or destroyed by the system responsive to the activities of the user. For instance, when a user walks into a living room, based on sensor information detecting the presence and/or activities of the user, various devices may be activated for providing services or information to or for the user. Each activity may spawn one or several documents corresponding to a device or devices. As stated, such devices may be stand-alone devices or device embedded in the environment. For example, much of the interaction in an ambient intelligence environment is derived from sensor data without the need for conscious intervention or even knowledge by the user. A common example of such a sensor is the one used near a door that automatically slides open when somebody stands in front of the door.
  • the document manager 1-11 drives the browser or is integrated therewith, and the browser manages controls the direct user interface with both documents. Accordingly, the timing and synchronization of media elements is controlled centrally because otherwise the total presentation could run out of synchronization because different devices use different clocks. Controller 1-17 performs other functions of the activity-related management system 1-1, such as overall coordination of the modules and interfacing with a user (not shown) via user interface/browser 1 -3.
  • the behavior-specified link includes an "add" command, which means that the retrieved document is added to the set of active documents currently managed by the document manager 1-11 of the context aware environment management system 1 - 1.
  • the links need not be behavior specified.
  • the documents contain triggers 2-12 and 2-22, respectively, that include links to other documents.
  • the documents may contain several such triggers, including multiple triggers with links to the same target documents. Documents may also contain no triggers. An instruction or behavior could also be associated with the link. Such a request for a document or behavior-specified link would be triggered when processing reaches that portion of the document and the behavior associated with the link would determine what action would be performed after the target document is accessed. For example, a given part of the target document could be retrieved and displayed, or the target document could be added or removed from the active list. Also, several links may be included in a trigger.
  • the document parser 1-12 parses the document based on actions of the user or based on events in the ambient intelligence environment that affect the document, and may determine what portions of the document are executed.
  • the trigger including pre-conditions and the link, may be added to or removed from a current trigger list as shown in Figure 4, column (e).
  • the following segment of code illustrates such a trigger and trigger behavior variable 2-21.
  • the link behavior specifies actions that are to be performed after the party document is accessed. For example, the document may be added to the active set of documents as in the current example, or the document may be removed from the active set if the link behavior specifies a different behavior.
  • the trigger behavior variable is shown in the first line of the code example.
  • the code beginning with ⁇ pre> until ⁇ /pre> is a pre-condition section of the trigger.
  • the pre-condition section includes the conditions that must be satisfied if the link of the trigger is to be traversed after it has been placed onto the current trigger list monitored by the context trigger list manager 1-14.
  • This code segment comprising a trigger and trigger behavior variable may be provided in the same document as the first code segment containing this trigger or may be contained in a different document.
  • the trigger including the link and the precondition segment will be added to the current trigger list.
  • this trigger will be removed from the current trigger list.
  • this trigger behavior specifier provides for a much more controlled, balanced way of trigger control and keeps smaller the active triggers that the system monitors.
  • the list of triggers is kept small because end- users set and retract triggers in the documents through their interactions with the environment and devices in the environment, but the trigger add and remove commands in the documents have been specified by the designer of the documents. Because multiple documents can be active at any time, one document can set a trigger that causes another document to be added or deleted from the active document set, as explained above with regard to the behavior specified links.
  • Figure 4 is a flow diagram showing aspects of system states as necessary for the present example.
  • Column (a) of Figure 4 shows the sensors/outputs arranged in the environment
  • column (b) shows status of the display document
  • column (c) shows status of the telephone document
  • column (d) shows status of the phone ringing document
  • column (e) shows at various moments in time the set of context triggers monitored on the current trigger list by the context trigger list manager 1-13 of the context aware environment management system 1 -1.
  • Sensor 1 shown for example as Sensor 1-51 of Figure 1 is activated when a user walks into a living room, shown at Sl of Figure 3 A and at column (a), row 3 of Fig. 4.
  • Various types of such sensors are known, such as light-sensitive sensors.
  • the current trigger list still contains no triggers.
  • Fig. 4 at (e2) shows that the current trigger list is empty at this point ⁇ null ⁇ .
  • This example may be an over simplification, since in most ambient intelligence systems, there would likely to be one or more triggers on the current trigger list at this point. However, this illustration suffices for the present discussion.
  • the activity/event detector 1-15 of the context aware environment management system 1-1 is provided a signal from the sensor, via a network 1-9 or directly and thus recognizes that a user is present in the living room and the document manager 1-1 activates a telephone document 2-2 of Figure 2, at S2 of Figure 3 A and at (c4) of Figure 4.
  • a living room user entry document (not shown) may be selected when a user's presence is detected, which can activate the telephone document 2-2 of Figure 2 and other documents.
  • document parser 1-12 parses the telephone document 2-2.
  • a context trigger 2-22 with the trigger behavior variable 2, 2-21 is reached, as shown at (c5) of Figure 4.
  • context trigger list manager 1-13 adds the context trigger 2-22 to the current trigger list (e), as shown at e4 of Figure 4. From this point forward, context trigger list manager 1-13 monitors periodically the phone ring trigger contained on the current trigger list as part of monitoring all of the context triggers on the current trigger list, and if the context trigger's preconditions are satisfied, then the link in the trigger will be traversed. It will be understood that a context trigger may include more than one link, and thus have more than one document as its target.
  • an incoming telephone call is detected.
  • the modem might sense an incoming call signal and transmit an indication of the signal to activity/event detector 1-15.
  • Context trigger list manager 1-13 monitors the current trigger list and now identifies the preconditions of the phone ring trigger 2-22 as having been satisfied. Accordingly, link handler 1-14 of Figure 4 traverses the link to the phone ringing document 2-3 and at (d8) of Figure 4, the telephone ringing document 2-3 of Figure 2 is selected (that is, the telephone ringing document 2-3 is added to the active document set shown in Figure 2).
  • Document manager 1-11 of Figure 1 manages the phone ringing document 2-3 in the browser module 2-9 of the server interface 1-16 of the context aware environment management system 1 -1.
  • the phone ringing document 2-3 may be nothing more than a description of one or more audio clips of a telephone sound.
  • the speaker or a bell provides a telephone ring.
  • the telephone ringing document 2-3 may then be removed from the set of active documents managed by the document manager 1-11.
  • the de-selection of the phone ringing document is shown at dlO of Figure 4.
  • sensor II is activated as a user sits down in a chair at a display.
  • the chair may be a desk on which a computer monitor is the display.
  • Activity/event detector 1-15 of context aware environment management system 1-1 activates the display document 2-1 of Fig. 2, as shown at (bl2) of Figure 4.
  • a living room manager document may have added a trigger (not shown) to the current trigger list, which trigger may have contained a link to the display document 2-1, which link is activated when the sensor II condition is satisfied.
  • document manager 1-11 adds the display document 2-12 to the set of currently active documents shown in browser module 2-9 of Figure 2.
  • the display document 2-1 is parsed by document parser 1-12 and at bl3 of Figure 4, the context trigger 2-12 is reached in the display document 2-1 of Fig. 2, as shown at S8 of Figure 3a.
  • Context trigger list manager 1-13 of Fig. 1 adds the display trigger to the current trigger list as shown at S9 of Figure 3a, since document parser 1-12 determines that the trigger behavior variable 1, 2-11, specifies an "add" behavior.
  • (el 4) of Figure 4 now shows that the current trigger list contains the phone ring trigger and the display trigger.
  • the display device is activated, as shown at al5 of Figure 4, when a section of the display document 2-1 is parsed which specifies that the display device is to be turned on.
  • Sensor II is activated when the user leaves the chair by the display as shown at S 14 of Figure 3b and at (a21) Figure 4.
  • Activity/event detector 1-15 detects the sensor input of sensor II, and the display document section that specifies displaying wallpaper on the display is now reached at S 15.
  • a phone ring trigger is reached in the display document, such that the phone ring trigger has an associated trigger behavior variable including an "add" specifier, which is interpreted by document parser 1-12.
  • context trigger list manager 1-13 adds the phone ring trigger back to the current trigger list as shown at (e23) of Figure 4 and at Sl 6 of Figure 3b.
  • Processing is stopped at S17 until further events are input to activity/event detector 1-15, such that the context trigger list manager 1-13 in continuing to monitor the current trigger list determines that preconditions of a context trigger are satisfied and invokes link handler 1-14 to traverse the link included in the trigger.

Abstract

A context aware environment management system (1-1), method and processor- readable medium are disclosed. The system may include a document parser (1-12) for parsing an active document (2-2) that includes setting information for user activity, the active document containing a trigger-behavior variable (2-21) and a context trigger (2-22) that includes a link to at least one target document that includes setting information for user activity and pre-conditions for the link. A list (e) of active context triggers is maintained and triggers are added when the trigger-behavior variable indicates that the context trigger is to be added. The list of active context triggers is monitored, and only the links of context triggers included on the list whose pre-conditions are satisfied is traversed. Further, the document parser (1-12) could be configured to parse a second such document (2-1) with additional or the same trigger (2-12) associated with a trigger-behavior variable (2-11).

Description

ACTIVITY-RELATED DOCUMENT MANAGEMENT
The present invention relates to the field of context aware systems, ambient intelligence environment systems, management of documents for managing user activity- related devices, and behavior-specified linking in browsers.
In recent years, more and more devices have become "wired" or connected via a network or via a central hub, making it possible to manage the functioning of these devices in an organized way. Home networks, in which household appliances and other consumer devices, are connected and may be remotely controlled, have also made begun to make their mark. Also, context aware systems and ambient intelligence systems have emerged which provide a digital environment that is sensitive, adaptive, and responsive to the presence of the people in the environment. In an ambient intelligence system, electronic devices are embedded in the furniture, clothing or other parts of the environment, and based on the user's activities detected by sensors of the system, devices are activated and services and information are delivered for the user.
In ambient intelligence environments, a user may be simultaneously using several networked devices, such that it would be desirable for the user's actions or activities with respect to one device or with respect to one activity to affect a second device or second activity. Also, more than one person often shares the environment, which causes activities of one user to affect the environment and thereby impact the second user sharing the environment. For example, in an ambient intelligence browser, the user may activate or control the operation or state of a device other than the device with which the user is directly interfaced or engaged. Such a browser may provide an interface to a user of an active document with a link, the document corresponding to a first user activity, and additional active documents corresponding to other activities of the user and the devices associated with those other activities. That is, each activity of a user could be represented by a document and the ambient intelligence browser can at any time support 0 to N activities (and corresponding documents) as active, N being a positive integer.
Each document describes how the presentation is rendered on one or more devices, that is where multimedia elements are to be presented in space (on which device and on which part of the screen, for example) and in time (some multimedia objects begin to play when others have stopped). An example of a document language that supports this is the SMIL language defined by the W3C (http://www.w3.org/AudioVideo/). The document also describes how users can interact with the application by means of forms that describe user interface widgets such as submit buttons, dropdown lists, text fields or the like. An example of a document language that supports this is XForms (http://www.w3.org/MarkUp/Forms/). XForms is intended to be used in conjunction with a web presentation language such as HTML or SMIL. In addition to this functionality, the document has linking functionality.
XLink (http://www.w3.org/XML/Linking) is a language for describing links and, like XForms, is meant to be used in conjunction with a presentation language like HTML or SMIL. XLink (http://www.w3.org/XML/Linking) defines a "show" attribute that is also used to communicate the desired presentation of the ending resource (end point of the link; the music playing document) on traversal from the starting resource (the begin point of the link; the webcam document). However, in XLink, and in presentation languages like HTML and SMIL for which XLink was meant, only one document at any given time is active. In such systems, only one document has focus.
Co-pending U.S. Application Provisional Application No. 60/ 632,138, filed on December 1, 2004, incorporated in full by reference herein, discloses a system and method of managing behavior-specified links in a multiple active document environment including, providing for a set of documents to be active simultaneously, each document specifying setting information for a user activity. According to this method, several documents may by simultaneously active, such that a behavior-specified link command included in a first document is processed, and this command affects at least a second document of the active document set.
Most context-aware systems either use a fixed list of triggers or lay the burden on the end-user of controlling which triggers are active. In the first case, this often means that whenever a trigger is activated (when all its preconditions have become true), the action specified by the trigger is performed, for example, the link is traversed. However, the list of active triggers changes depending on the situation arising in the environment. For example, sometimes a user wants to be disturbed by a ringing telephone, sometimes the user does not. This means that many different triggers would have to be defined. However, the longer the list of triggers being monitored by the system, the greater the chance triggers are activated that are not supposed to be activated. In other systems, users must consciously set and retract triggers. The designer of the system forces the end-user to manage the set of triggers. But this soon gets too difficult for end-users and the aim of achieving a truly ambient intelligence is hindered.
WO Patent 2003/101045 is directed to a physical hypertext language, a way of describing how real world devices can be controlled by a declarative language. WO Patent 2003/101045 does not describe a trigger language module that could be used in a hypertext language that describes the interaction of the user with devices in the physical world or monitoring conditions or pre-conditions associated with such triggers.
U.S. Patent Application Publication No. 2003/0182474 is directed to a way of mapping high-level device controls on low-level device implementations in a generic way, including receiving a high-level device control command, generating a specific device control script based on a device interconnect model and the high-level device control command received, interpreting the generic device control script, and transmitting specific device control commands to a device through a device interface. This reference does not teach that a user without conscious effort or even awareness can set or retract trigger modules for applications.
U.S. Patent 5,611,050 is directed to a system that grants interaction requests that have locational and contextual attributes that are consistent with specified interaction policies and denies those requests that are inconsistent. The invention electronically monitors contextual information concerning user and machines, including state and location information including proximity, and traverses links if the conditions for the link are valid. U.S. Patent 5,611 ,050 does not disclose a system in which the system controls or provides the event information for traversing the link without conscious effort by the user of the system.
U.S. Patent Application Publication No. 2002/0035404 is directed to a set-top box that controls the operation of certain devices designated in the script in a way somewhat similar to the way described in WO Patent 2003/101045. The patent describes that each of the scripts is initiated based on a corresponding triggering mechanism that can be set by the user via a graphical user interface (GUI). The user enters the trigger and its firing preconditions in the GUI. The system then waits for the preconditions to become true and fires the trigger. U.S. Patent Application Publication No. 2002/0035404 does not disclose a context aware system in which the user does not consciously set/deactivate triggers.
A context aware environment management system, method and processor- readable medium are disclosed. The system may include a document parser configured to parse an active document that specifies setting information for user activity, the active document containing a trigger-behavior variable and a context trigger that includes a link to at least one target document specifying setting information for user activity and preconditions for the link; a context trigger list manager configured to add the context trigger to a list of active context triggers when the trigger-behavior variable indicates that the context trigger is to be added; such that the context trigger list manager configured to monitor the list of active context triggers, and to traverse only the links of context triggers included on the list whose pre-conditions are satisfied.
Further in this system, the document parser could be configured to parse a second document specifying setting information for user activity containing the context trigger and a second trigger-behavior variable associated with the context trigger; and the context trigger list manager may be deployed to remove the context trigger from the list when the second trigger-behavior variable indicates that the context trigger is to be removed from the list.
Also, in this system, the document parser may reach the context trigger in the active document based on an event related to the user activity specified by the document. Moreover, this system may be an ambient intelligence engine that includes a browser and the link may be a behavior-specified link managed by the browser.
According to this system the document manager of the ambient intelligence engine may be able simultaneously to maintain at least one additional document as an active document.
Figure 1 illustrates a schematic view of a context aware environment management system according to an embodiment of the present invention.
Figure 2 illustrates a browser module according to an aspect of the present invention providing user interface with active documents.
Figures 3A and 3B illustrate a flowchart for an operation of a context aware environment management system according to an aspect of the present invention.
Figure 4 is a flow diagram showing aspects of system states according to an embodiment of the present invention. The following discussion and the foregoing figures describe embodiments of Applicant's invention as best understood presently by the inventor however, it will be appreciated that numerous modifications of the invention are possible and that the invention may be embodied in other forms and practiced in other ways without departing from the spirit of the invention. Further, features of embodiments described may be omitted, combined selectively, or as a whole, with other embodiments, or used to replace features of other embodiments, or parts thereof, without departing from the spirit of the invention. The figures and the detailed description are therefore to be considered as an illustrative explanation of aspects of the invention, but should not be construed to limit the scope of the invention.
As shown in Figure 1, the context aware environment management system 1-1 includes several modules, which will be described below. Modules of the activity-related document management system 1 -1, or portions thereof, and/or the activity-related document management system as a whole, may be comprised of hardware, software, firmware, or a combination of the foregoing, however some modules may be comprised of hardware for example, while other modules may be comprised of software, firmware or a combination thereof.
It is to be understood that modules of the activity-related document management system need not all be located or integrated with the same device. A distributed architecture is also contemplated for the activity-related document management system, which may "piggy-back" off of suitable modules provided by existing devices.
The following description will refer to a context aware environment management system 1-1 that is physically integrated with or connected to a database 1-2 via a wired or wireless connection thereto. The database 1 -2 may be embodied on a storage device such as on a hard drive of a personal computer, a personal video recorder, an entertainment system, an electronic organizer, a personal handheld device, a Jaz drive, or may be embodied as a commercial storage facility, such as a disk drive. For example, a commercial storage facility may include a storage space provided by a service provider, or residing in the network. It will be understood that the database 1-2 may include several storage devices that are connected, such that organization or grouping of content items on two or more of such devices is possible. For example, the data may be distributed over devices in a peer-to-peer network. By way of example, data derived from sensors may thus be handled. It will further be understood that the database may be understood to include one or more storage media, such as disks, including CDs, DVDs, zip disks, floppy disks, data cartridges, or the like, which can be loaded onto and retrieved by the database 1-2. However, it will be understood that the context aware environment management system 1- 1 is also capable of retrieving content via a network 1-9, such as a LAN, WAN, the internet, or the like, and that the database may be remotely connected, such as via a network, including the internet.
As shown in Figure 1, context aware environment management system 1-1 includes a document manager 1-11 which manages documents currently active in the system. For example, context aware environment management system 1-1 may include a browser or be logically connected to a browser, or browser module, as shown in Figure 2, including an ambient intelligent browser. The ambient intelligence browser may also be understood as comprising a module or network node outside of the context aware environment management system 1-1, for example as a kind of navigation browsing assistant. Alternatively, the context aware environment management system 1-1, including the document manager 1-11, may be understood as the ambient intelligence browser plus the extra functionality discussed herein and shown as the context aware environment management system 1-1.
Also, device 1-52 and sensor 1-51 may be connected in a wired or wireless mode directly to the context aware environment management system 1-1 or to a device physically integrated with the context aware environment management system 1-1, or as shown in Fig. 1 may be connected to the context aware environment management system 1-1 via a network 1-9, such as wired home network application, or other type or wired or wireless network, including a LAN, a WAN or the internet.
As shown in Figure 2, according to an aspect of the present invention the context aware environment management system 1-1 may provide for several simultaneously active documents, shown as display document, 2-1, telephone document, 2- 2, and phone ringing document 2-3. Such documents may control devices associated with activities performed by the user, such as CRT monitor or flat panel display or the like controlled by display document 2-1, telephone function controlled by telephone document 2-2, and phone ringing functions controlled by the phone ringing document 2-3. Several sets of such documents may be simultaneously active, each set corresponding, for example, to a different user. The browser module shown in Figure 2 may be integrated with the document manager 1-11 of the context aware environment management system 1-1 or be a separate module, as discussed.
According to an aspect of the present invention, in an ambient intelligence environment, a document or a set of several documents may be created or destroyed by the system responsive to the activities of the user. For instance, when a user walks into a living room, based on sensor information detecting the presence and/or activities of the user, various devices may be activated for providing services or information to or for the user. Each activity may spawn one or several documents corresponding to a device or devices. As stated, such devices may be stand-alone devices or device embedded in the environment. For example, much of the interaction in an ambient intelligence environment is derived from sensor data without the need for conscious intervention or even knowledge by the user. A common example of such a sensor is the one used near a door that automatically slides open when somebody stands in front of the door.
According to an aspect of the present invention, the document manager 1-11 drives the browser or is integrated therewith, and the browser manages controls the direct user interface with both documents. Accordingly, the timing and synchronization of media elements is controlled centrally because otherwise the total presentation could run out of synchronization because different devices use different clocks. Controller 1-17 performs other functions of the activity-related management system 1-1, such as overall coordination of the modules and interfacing with a user (not shown) via user interface/browser 1 -3.
The instruction:
<link id="linkl" behaviour="add" from="." to=http : //anengine : 8080/query?type=music_playing/> is an illustrative example of a format for a type of behavior-specified link that specifies that the reference to the document specified by its URL is to be added to the set of active documents. In the example set forth in the above-enumerated code, the behavior-specified link includes an "add" command, which means that the retrieved document is added to the set of active documents currently managed by the document manager 1-11 of the context aware environment management system 1 - 1. However, the links need not be behavior specified. As shown in Fig. 2, the documents contain triggers 2-12 and 2-22, respectively, that include links to other documents. The documents may contain several such triggers, including multiple triggers with links to the same target documents. Documents may also contain no triggers. An instruction or behavior could also be associated with the link. Such a request for a document or behavior-specified link would be triggered when processing reaches that portion of the document and the behavior associated with the link would determine what action would be performed after the target document is accessed. For example, a given part of the target document could be retrieved and displayed, or the target document could be added or removed from the active list. Also, several links may be included in a trigger.
The document parser 1-12 parses the document based on actions of the user or based on events in the ambient intelligence environment that affect the document, and may determine what portions of the document are executed. When a context trigger is reached, the trigger, including pre-conditions and the link, may be added to or removed from a current trigger list as shown in Figure 4, column (e).
The following segment of code illustrates such a trigger and trigger behavior variable 2-21.
<trigger id="telephoneSound behaviour="add" persistence="true"> <pre>
<stage id="living" location="home>
<performance id="telephoneRinging">
<prop id="telephonel" capability="voice I/O"/>
</performance> </stage> </pre>
<link behaviour="add" id="linkl" to="http: //anengine: 8080/query?type=telephoneSound"/> </trigger>
The section of code after </pre> is the link section beginning with <iink which specifies that the document following the to= specifier is to be accessed when the link is traversed. As explained above, the link behavior specifies actions that are to be performed after the party document is accessed. For example, the document may be added to the active set of documents as in the current example, or the document may be removed from the active set if the link behavior specifies a different behavior.
The trigger behavior variable is shown in the first line of the code example. The segment: behaviour="add" specifies that the trigger that follows, including the link and the preconditions for the link, are to be added to the current trigger list as a currently active trigger. The specifier id=telephoneSound identifies this trigger for the context aware environment management system 1-1. This will be especially useful for tracking the trigger once it is on the current trigger list.
The code beginning with <pre> until </pre> is a pre-condition section of the trigger. The pre-condition section includes the conditions that must be satisfied if the link of the trigger is to be traversed after it has been placed onto the current trigger list monitored by the context trigger list manager 1-14.
Similarly, the following code segment shows that this same trigger can be removed by specifying in the trigger a behavior variable indicating that the trigger is to be removed, such as by including the specifier behaviour=" remove" as shown below:
<trigger id="telephoneSound" behaviour="remove" persistence="true"> <pre>
<stage id="living" location="home>
<performance id="telephoneRinging">
<prop id="telephonel" capability="voice I/O"/> </performance> </stage> </pre>
<link behaviour="add" id="linkl" to="http: //anengine: 8080/query?type=telephoneSound"/> </trigger>
This code segment comprising a trigger and trigger behavior variable may be provided in the same document as the first code segment containing this trigger or may be contained in a different document. In response to the execution by the document parser 1- 12 of the first code segment, the trigger including the link and the precondition segment will be added to the current trigger list. In response to the execution by the document parser 1-12 of the second code segment, this trigger will be removed from the current trigger list.
According to an aspect of the present invention, this trigger behavior specifier provides for a much more controlled, balanced way of trigger control and keeps smaller the active triggers that the system monitors. The list of triggers is kept small because end- users set and retract triggers in the documents through their interactions with the environment and devices in the environment, but the trigger add and remove commands in the documents have been specified by the designer of the documents. Because multiple documents can be active at any time, one document can set a trigger that causes another document to be added or deleted from the active document set, as explained above with regard to the behavior specified links.
Figure 4 is a flow diagram showing aspects of system states as necessary for the present example. Column (a) of Figure 4 shows the sensors/outputs arranged in the environment, column (b) shows status of the display document, column (c) shows status of the telephone document, column (d) shows status of the phone ringing document, and column (e) shows at various moments in time the set of context triggers monitored on the current trigger list by the context trigger list manager 1-13 of the context aware environment management system 1 -1.
An operation of an embodiment of the context awareness environment management system 1-1 will now be given with reference to an example involving a single user and a small number of active documents. It will be understood of course that this simple example using a few active documents and several context triggers is provided only to illustrate aspects of the Applicants invention.
Sensor 1, shown for example as Sensor 1-51 of Figure 1, is activated when a user walks into a living room, shown at Sl of Figure 3 A and at column (a), row 3 of Fig. 4. Various types of such sensors are known, such as light-sensitive sensors. At this point, the current trigger list still contains no triggers. Fig. 4 at (e2) shows that the current trigger list is empty at this point { null } . (This example may be an over simplification, since in most ambient intelligence systems, there would likely to be one or more triggers on the current trigger list at this point. However, this illustration suffices for the present discussion.)
The activity/event detector 1-15 of the context aware environment management system 1-1 is provided a signal from the sensor, via a network 1-9 or directly and thus recognizes that a user is present in the living room and the document manager 1-1 activates a telephone document 2-2 of Figure 2, at S2 of Figure 3 A and at (c4) of Figure 4. For example, a living room user entry document (not shown) may be selected when a user's presence is detected, which can activate the telephone document 2-2 of Figure 2 and other documents. At S3, after the telephone document 2-2 is launched, document parser 1-12 parses the telephone document 2-2. At S3, as the document parser 1-12 processes the telephone document 2-2 shown in Fig. 2, a context trigger 2-22 with the trigger behavior variable 2, 2-21 is reached, as shown at (c5) of Figure 4.
At this point, the context trigger a 2-22 is read and interpreted by document parser 1-12. Since the trigger behavior variable 2, 2-21, shows an "add" behavior for the context trigger, at S4 context trigger list manager 1-13 adds the context trigger 2-22 to the current trigger list (e), as shown at e4 of Figure 4. From this point forward, context trigger list manager 1-13 monitors periodically the phone ring trigger contained on the current trigger list as part of monitoring all of the context triggers on the current trigger list, and if the context trigger's preconditions are satisfied, then the link in the trigger will be traversed. It will be understood that a context trigger may include more than one link, and thus have more than one document as its target.
At S5 of Figure 3a and at a7 of Figure 4, an incoming telephone call is detected. For instance, in voice over IP telephone application, the modem might sense an incoming call signal and transmit an indication of the signal to activity/event detector 1-15. Context trigger list manager 1-13 monitors the current trigger list and now identifies the preconditions of the phone ring trigger 2-22 as having been satisfied. Accordingly, link handler 1-14 of Figure 4 traverses the link to the phone ringing document 2-3 and at (d8) of Figure 4, the telephone ringing document 2-3 of Figure 2 is selected (that is, the telephone ringing document 2-3 is added to the active document set shown in Figure 2).
Document manager 1-11 of Figure 1 manages the phone ringing document 2-3 in the browser module 2-9 of the server interface 1-16 of the context aware environment management system 1 -1. The phone ringing document 2-3 may be nothing more than a description of one or more audio clips of a telephone sound. At (a9) of Figure 4 and at S6 of Figure 3a, the speaker or a bell provides a telephone ring. The telephone ringing document 2-3 may then be removed from the set of active documents managed by the document manager 1-11. The de-selection of the phone ringing document is shown at dlO of Figure 4. However, this does not mean that the phone ring trigger is necessarily removed from the current trigger list, and for purposes of illustration, in the present example, this context trigger is kept on the current trigger list until it is removed pursuant to a trigger remove behavior specified in a parsed section of an active document. At (al 1) of Figure 4 and at S7 of Figure 3 a, sensor II is activated as a user sits down in a chair at a display. For example, the chair may be a desk on which a computer monitor is the display. Activity/event detector 1-15 of context aware environment management system 1-1 activates the display document 2-1 of Fig. 2, as shown at (bl2) of Figure 4. For example, a living room manager document (not shown) may have added a trigger (not shown) to the current trigger list, which trigger may have contained a link to the display document 2-1, which link is activated when the sensor II condition is satisfied. As a result, document manager 1-11 adds the display document 2-12 to the set of currently active documents shown in browser module 2-9 of Figure 2.
The display document 2-1 is parsed by document parser 1-12 and at bl3 of Figure 4, the context trigger 2-12 is reached in the display document 2-1 of Fig. 2, as shown at S8 of Figure 3a. Context trigger list manager 1-13 of Fig. 1 adds the display trigger to the current trigger list as shown at S9 of Figure 3a, since document parser 1-12 determines that the trigger behavior variable 1, 2-11, specifies an "add" behavior. Accordingly, (el 4) of Figure 4 now shows that the current trigger list contains the phone ring trigger and the display trigger. The display device is activated, as shown at al5 of Figure 4, when a section of the display document 2-1 is parsed which specifies that the display device is to be turned on.
At SlO of Figure 3 a and at (bl6) of Figure 4, as the document parser 1-12 continues parsing the display document 2-1, a phone ring trigger with trigger-behavior variable specifying "remove" behavior for the trigger is reached. Accordingly, as shown at SI l of Figure 3b, context trigger list manager 1-13 removes the phone ring trigger from the current trigger list. Figure 4, (el 7), shows that the current trigger list now contains only the display trigger. As a result, as context trigger list manager 1-13 continues monitoring of the current trigger list from time to time, even when the preconditions set in the phone ring trigger are satisfied, the link included in the phone ring trigger will not be traversed. That is, as shown at S12 of Figure 3a and at (al8) of Figure 4, when an incoming telephone call is detected by activity/event detector 1-15, the phone ringing document 2-3 of Figure 2 will not be activated, since no link to the phone ringing document will be traversed. As result, as shown S 13 and at (a20), no telephone ring is sounded.
Sensor II is activated when the user leaves the chair by the display as shown at S 14 of Figure 3b and at (a21) Figure 4. Activity/event detector 1-15 detects the sensor input of sensor II, and the display document section that specifies displaying wallpaper on the display is now reached at S 15.
Further, at S 16, a phone ring trigger is reached in the display document, such that the phone ring trigger has an associated trigger behavior variable including an "add" specifier, which is interpreted by document parser 1-12. Accordingly, context trigger list manager 1-13 adds the phone ring trigger back to the current trigger list as shown at (e23) of Figure 4 and at Sl 6 of Figure 3b. Processing is stopped at S17 until further events are input to activity/event detector 1-15, such that the context trigger list manager 1-13 in continuing to monitor the current trigger list determines that preconditions of a context trigger are satisfied and invokes link handler 1-14 to traverse the link included in the trigger.
While there has been shown and described what is considered to be preferred embodiments of the invention, it will, of course, be understood that various modifications and changes in form or detail could readily be made without departing from the spirit of the invention. It is therefore intended that the invention be not limited to the exact forms described and illustrated, but should be constructed to cover all modifications that may fall within the scope of the appended claims.

Claims

CLAIMS:
1. A method of managing a context aware environment, the method
comprising: parsing (S3) an active document (2-2) that specifies setting information for user activity, the active document containing a trigger-behavior variable (2-21) and a context trigger (2-22) that includes at least one pre-condition for a link to at least one target document specifying setting information for user activity and the link to the target document; adding (S4) the context trigger to a list of active context triggers when the trigger-behavior variable indicates that the context trigger is to be added; and monitoring the list of active context triggers by a context aware system (1- 1), and traversing (S5) only the links of context triggers included on the list of active context triggers whose pre-conditions are satisfied.
2. The method of claim 1, further comprising: parsing (S7) a second document (2-1) that specifies setting information for user activity and that contains the context trigger (2-12) and a second trigger-behavior variable (2-11) associated with the context trigger; and removing (Sl 1) the context trigger from the list of active context triggers when the second trigger-behavior variable indicates that the context trigger is to be removed from the list of active context triggers.
3. The method of claim 1, wherein the context aware system is an ambient intelligence engine that includes a browser (2-9) and the link is a behavior-specified link managed by the browser.
4. The method of claim 3, wherein the ambient intelligence engine simultaneously maintains at least one additional document as an active document.
5. The method of claim 3, wherein the behavior-specified link links to a plurality of documents.
6. The method of claim 1, wherein the context trigger in the active document is reached based on an event related to the user activity specified by the document.
7. The method of claim 2, wherein the first document and the second document are the same document.
8. The method of claim 2, wherein the first document and the second document are different documents.
9. A context aware environment management system comprising: a document parser (1-12) configured to parse an active document (2-2) that specifies setting information for user activity, the active document containing a trigger- behavior variable (2-21) and a context trigger (2-22) that includes pre-conditions for a link to at least one target document specifying setting information for user activity and the link to the at least one target document; a context trigger list manager (1-13) configured to add the context trigger to a list of active context triggers when the trigger-behavior variable indicates that the context trigger is to be added; and said context trigger list manager configured to monitor the list of active context triggers, and to traverse (S5) only the links of context triggers included on the list whose pre-conditions are satisfied.
10. The system of claim 9, wherein: said document parser (1-12) is configured to parse a second document (2-1) that specifies setting information for user activity containing the context trigger (2-12) and a second trigger-behavior variable (2-11) associated with the context trigger; and said context trigger list manager (1-13) is configured to remove the context trigger from the list when the second trigger-behavior variable indicates that the context trigger is to be removed from the list.
11. The system of claim 9, wherein the document parser reaches the context trigger in the active document based on an event related to the user activity specified by the document.
12. The system of claim 9, wherein the system is an ambient intelligence engine that includes a browser (2-9) and the link is a behavior-specified link managed by the browser.
13. The system of claim 12, wherein a document manager of the ambient intelligence engine simultaneously maintains at least one additional document as an active document.
14. A processor readable medium incorporating a program of instructions executable by a machine, the program of instructions for managing a context aware environment, the program comprising: means for parsing (1-12) an active document that specifies setting information for user activity, the active document containing a trigger-behavior variable and a context trigger that includes pre-conditions for a link to at least one target document specifying setting information for user activity and the link to the target document; means for adding (1 -13) the context trigger to a list of active context triggers when the trigger-behavior variable indicates that the context trigger is to be added; and means for monitoring (1-13) the list of active context triggers by a context aware system, and traversing only the links of context triggers included on the list of active context triggers whose pre-conditions are satisfied.
15. The medium incorporating the program of instructions of claim 14, comprising: means for parsing (1-12) a second document that specifies setting information for user activity and the context trigger and a second trigger-behavior variable associated with the context trigger; and means for removing (1-13) the context trigger from the list of active context triggers when the second trigger-behavior variable indicates that the context trigger is to be removed from the list of active context triggers.
16. The medium incorporating the program of instructions of claim 14, wherein the instructions provide an ambient intelligence engine that includes a browser and the link is a behavior-specified link managed by the browser.
17. The medium incorporating the program of instructions of claim 16, wherein the ambient intelligence engine simultaneously maintains at least one additional document as an active document.
18. The medium incorporating the program of instructions of claim 14, wherein the context trigger in the active document is reached based on an event related to the user activity specified by the document.
19. The medium incorporating the program of instructions of claim 14, wherein the first document and the second document are the same document.
20. The medium incorporating the program of instructions of claim 14, wherein the first document and the second document are different documents.
PCT/IB2006/051229 2005-04-21 2006-04-20 Activity-related document mangement WO2006111935A1 (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
US67354005P 2005-04-21 2005-04-21
US60/673,540 2005-04-21

Publications (1)

Publication Number Publication Date
WO2006111935A1 true WO2006111935A1 (en) 2006-10-26

Family

ID=36636658

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/IB2006/051229 WO2006111935A1 (en) 2005-04-21 2006-04-20 Activity-related document mangement

Country Status (1)

Country Link
WO (1) WO2006111935A1 (en)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP2194472A1 (en) * 2008-12-02 2010-06-09 Palo Alto Research Center Incorporated Context and activity-driven content delivery and interaction
WO2017031425A1 (en) * 2015-08-19 2017-02-23 Google Inc. Physical knowledge action triggers

Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5611050A (en) * 1993-12-03 1997-03-11 Xerox Corporation Method for selectively performing event on computer controlled device whose location and allowable operation is consistent with the contextual and locational attributes of the event

Patent Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5611050A (en) * 1993-12-03 1997-03-11 Xerox Corporation Method for selectively performing event on computer controlled device whose location and allowable operation is consistent with the contextual and locational attributes of the event

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
AL-BIN-ALI F: "Design Principles for Inducing Reactivity in Ubiquitous Environments", PERVASIVE SERVICES, 2004. ICPS 2004. IEEE/ACS INTERNATIONAL CONFERENCE ON BEIRUT, LEBANON 19-23 JULY 2004, PISCATAWAY, NJ, USA,IEEE, 19 July 2004 (2004-07-19), pages 131 - 139, XP010760003, ISBN: 0-7695-2535-0 *
MIN HONG YUN ET AL: "Event-based multimedia object scheduling algorithm", ADVANCED COMMUNICATION TECHNOLOGY, 2004. THE 6TH INTERNATIONAL CONFERENCE ON PHOENIX PARK, KOREA FEB. 9-11, 2004, PISCATAWAY, NJ, USA,IEEE, vol. 2, 9 February 2004 (2004-02-09), pages 735 - 740, XP010703054, ISBN: 89-5519-119-7 *

Cited By (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP2194472A1 (en) * 2008-12-02 2010-06-09 Palo Alto Research Center Incorporated Context and activity-driven content delivery and interaction
US8489599B2 (en) 2008-12-02 2013-07-16 Palo Alto Research Center Incorporated Context and activity-driven content delivery and interaction
CN107038197A (en) * 2008-12-02 2017-08-11 帕洛阿尔托研究中心公司 The content transmission and interaction of situation and activity-driven
WO2017031425A1 (en) * 2015-08-19 2017-02-23 Google Inc. Physical knowledge action triggers
GB2554269A (en) * 2015-08-19 2018-03-28 Google Llc Physical knowledge action triggers

Similar Documents

Publication Publication Date Title
US8850322B2 (en) Customized diagrammatic view of a network topology
US20070250191A1 (en) System and method for configuring and maintaining individual and multiple environmental control units over a communication network from an administration system
US20120079507A1 (en) Management and marketplace for distributed home devices
US20120167047A1 (en) System and method for automatic creation of web content for mobile communicators
US7788340B2 (en) System and method for event based interportlet communications
KR101961741B1 (en) Method for activating and communication widget
TW200841244A (en) Managing web page links using structured data
US20090119595A1 (en) Method And System For Presenting A Transition Effect Between Remotely-Sourced Content In A Browser
CN103425483A (en) User customization of menus
WO2012075879A1 (en) Method for implementing client software for web application
JP2003015793A (en) Method and system for dynamically changing and displaying information to be monitored on network on monitor screen and user interface player program for realizing the same system
JPWO2014042158A1 (en) Content display device, content display system, content display method, and content display program
US20080228904A1 (en) Home Gateway Architecture and State Based Distributed System and Method
CN103425480B (en) The method and apparatus of configuration and the management of menu
CN109977260B (en) Video acquisition method, device, system, electronic equipment and storage medium
US20120233560A1 (en) Method for managing widgets in an electronic device to improve the user experience of the device
WO2006111935A1 (en) Activity-related document mangement
Loke Logic programming for context-aware pervasive computing: Language support, characterizing situations, and integration with the web
CN103425566A (en) Menu usage statistics collection and reporting
JP2006285308A (en) Service providing system
KR20100039017A (en) Remote control apparatus using menu markup language
JP5014434B2 (en) Dynamic layout
Vildjiounaite et al. Context-dependent user modelling for smart homes
US20090150452A1 (en) Activity-related document managmenet
CN113391746B (en) Display equipment and multi-window focus control method

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application
NENP Non-entry into the national phase

Ref country code: DE

NENP Non-entry into the national phase

Ref country code: RU

WWW Wipo information: withdrawn in national office

Country of ref document: RU

122 Ep: pct application non-entry in european phase

Ref document number: 06727989

Country of ref document: EP

Kind code of ref document: A1

WWW Wipo information: withdrawn in national office

Ref document number: 6727989

Country of ref document: EP