WO2000036493A1 - Method and system for controlling presentation of information to a user based on the user's condition - Google Patents

Method and system for controlling presentation of information to a user based on the user's condition Download PDF

Info

Publication number
WO2000036493A1
WO2000036493A1 PCT/US1999/029952 US9929952W WO0036493A1 WO 2000036493 A1 WO2000036493 A1 WO 2000036493A1 US 9929952 W US9929952 W US 9929952W WO 0036493 A1 WO0036493 A1 WO 0036493A1
Authority
WO
WIPO (PCT)
Prior art keywords
user
information
output
presenting
computer
Prior art date
Application number
PCT/US1999/029952
Other languages
French (fr)
Inventor
Dan Newell
Kenneth H. Abbott, Iii
Original Assignee
Tangis Corporation
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Tangis Corporation filed Critical Tangis Corporation
Priority to AU21902/00A priority Critical patent/AU2190200A/en
Priority to DE69902487T priority patent/DE69902487T2/en
Priority to EP99966337A priority patent/EP1151372B1/en
Publication of WO2000036493A1 publication Critical patent/WO2000036493A1/en

Links

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F21/00Security arrangements for protecting computers, components thereof, programs or data against unauthorised activity
    • G06F21/60Protecting data
    • G06F21/606Protecting data by securing the transmission between two devices or processes
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F1/00Details not covered by groups G06F3/00 - G06F13/00 and G06F21/00
    • G06F1/16Constructional details or arrangements
    • G06F1/1613Constructional details or arrangements for portable computers
    • G06F1/163Wearable computers, e.g. on a belt
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F21/00Security arrangements for protecting computers, components thereof, programs or data against unauthorised activity
    • G06F21/60Protecting data
    • G06F21/62Protecting access to data via a platform, e.g. using keys or access control rules
    • G06F21/6218Protecting access to data via a platform, e.g. using keys or access control rules to a system of files or objects, e.g. local or distributed file system or database
    • G06F21/6245Protecting personal data, e.g. for financial or medical purposes
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/011Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/016Input arrangements with force or tactile feedback as computer generated output to the user
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F2221/00Indexing scheme relating to security arrangements for protecting computers, components thereof, programs or data against unauthorised activity
    • G06F2221/03Indexing scheme relating to G06F21/50, monitoring users, programs or devices to maintain the integrity of platforms
    • G06F2221/032Protect output to user by software means
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F2221/00Indexing scheme relating to security arrangements for protecting computers, components thereof, programs or data against unauthorised activity
    • G06F2221/21Indexing scheme relating to G06F21/00 and subgroups addressing additional information or applications relating to security arrangements for protecting computers, components thereof, programs or data against unauthorised activity
    • G06F2221/2149Restricted operating environment

Definitions

  • the present invention relates generally to computer program user interfaces, and more particularly to presenting information to a user based on the user's current condition
  • computers As computers become increasingly powerful and ubiquitous, users increasingly use their computers for a broad va ⁇ ety of tasks. For example, in addition to traditional activities such as running word processing and database applications, users increasingly use computers as an integral part of their daily lives. Programs to schedule activities, generate reminders, and provide rapid communication capabilities are becoming increasingly popular Moreover, computers are increasingly present during virtually all of a person's daily activities. For example, hand-held computer organizers (e.g., PDAs) are increasingly common, and communication devices such as portable phones are increasingly incorporating computer capabilities Thus, users may be presented with output information from one or more computers at any time.
  • PDAs personal digital assistants
  • the computer can safely present information to the user at any time, with minimal ⁇ sk that the user will fail to perceive the information or that the information will disturb the user in a dangerous manner (e g , by startling the user while they are using power machinery or by blocking their vision while they are moving with information sent to a head-mounted display)
  • a dangerous manner e g , by startling the user while they are using power machinery or by blocking their vision while they are moving with information sent to a head-mounted display
  • these assumptions about the prototypical user are not true, and users thus may not perceive output information (e g , failing to notice an icon or message on a hand-held display dev ice when it is holstered, or failing to hear audio information when in a noisy environment or when intensely concentrating)
  • some user activities may have a low degree of lnterruptibihty (t e , ability to safely interrupt the user) such that the user would prefer that the presentation of low-importance or of all information be deferred, or that information be presented in
  • wearable computers are designed to act as constant companions and intelligent assistants to a user, thus being available to receive input from the user at any time and to present output information to the user at any time
  • Wearable computers are typically strapped to the user's body or mounted in a holster, and may include a va ⁇ ety of both input and output devices
  • the close association of wearable computers to their users results in the wearable computer interacting with the user in virtually any social or business situation, and thus the likelihood of inapprop ⁇ ate output behavior increases
  • Some embodiments of the present invention provide a method and system for controlling presentation of information t a user based on the user's current condition
  • the system monitors the user and the user's environment, and creates and maintains an updated model of the current condition of the user
  • the user condition can include a va ⁇ ety of condition va ⁇ ables, including abstract concepts such as the user's current cognitive load, desired level of p ⁇ vacy for output information, and desired scope of audience for output information
  • the user condition can also include condition va ⁇ ables indicating physical charactenstics (e g , deafness) and physically observable charactenstics (e g , movement or proximity to another object) of the user
  • the system determines an approp ⁇ ate output device and an approp ⁇ ate format with which to present the information to the user, and then presents the output information
  • the system also receives desc ⁇ ption information about the output information that desc
  • the system presents output information to a user by first receiving information about a modeled characte ⁇ stic of the user which may include a modeled preference of the user for receiving sensitive information, a modeled indication of a current degree of interruptibilitv of the user, or a modeled preference of the user for an amount of people to perceive information presented by the computer
  • the system selects an output device capable of presenting the output information in accordance with the modeled characte ⁇ stic, and presents the output information on the selected output device in accordance with the modeled characte ⁇ stic
  • the system presents information to a user on one of multiple available output devices
  • the system monitors the user to collect information about a current state of the user, and then models a current user condition based on the collected information by determining a current level of p ⁇ vacy desired by the user that indicates a group of people allowed to perceive information presented by the computer, by determining a current scope of audience desired by the user that indicates how many people are intended to perceive information presented by the computer,
  • Figure 1 illustrates a user wea ⁇ ng a body-mounted computer executing an embodiment of the Condition-Dependent Output Supplier (CDOS) system of the present invention
  • CDOS Condition-Dependent Output Supplier
  • FIG. 2 is a block diagram illustrating the contents and information flow of an embodiment of the CDOS system
  • Figure 3 is an illustrative example of a model of a current user condition
  • Figure 4 is an illustrative example of a User Characte ⁇ zation Module
  • Figure 5 is an illustrative example of an Output Device Selector Module
  • Figure 6 is an exemplary flow diagram of an embodiment of the User Characte ⁇ zation routine
  • Figure 7 is an exemplary flow diagram of an embodiment of the Characte ⁇ ze User subroutine
  • Figure 8 is an exemplary flow diagram of an embodiment of the Output Device Selector routine.
  • Figure 9 is an exemplary flow diagram of an embodiment of the Format And Present Output Information subroutine.
  • a embodiment of the present invention provides a method and system for controlling presentation of information to a user based on the user's current condition.
  • the Condition-Dependent Output Supplier (CDOS) system monitors the user and the user's environment, and creates and maintains an updated model of the current condition of the user.
  • the CDOS system Upon receiving output information to be presented to the user (e.g., from an application program), the CDOS system determines an appropriate output device and an appropriate format with which to present the information to the user, and then presents the output information.
  • the CDOS system also receives description information about the output information that describes relevant factors for determining when and how to present the output information (e.g., the impo ⁇ ance and urgency of the output information, the consequences of the user not receiving or ignoring the output information, etc.).
  • description information about the output information that describes relevant factors for determining when and how to present the output information (e.g., the impo ⁇ ance and urgency of the output information, the consequences of the user not receiving or ignoring the output information, etc.).
  • the model of the user's current condition includes a variety of condition variables that represent information about the user and the user's environment at varying levels of abstraction.
  • information about the user at a low level of abstraction can include raw physiological data (e.g., heart rate and EKG) and geographic information (e.g., location and speed), while higher levels of abstraction may attempt to characterize or predict the user's physical activity (e.g., jogging or talking on a phone), emotional state (e.g., angry or puzzled), desired output behavior for different types of information (e.g., to present private family information so that it is perceivable only to myself and my family members), and cognitive load (i.e., the amount of attention required for the user's current activities).
  • information about the user at a low level of abstraction can include raw physiological data (e.g., heart rate and EKG) and geographic information (e.g., location and speed), while higher levels of abstraction may attempt to characterize or predict the user's physical activity (e.g., jogg
  • the model can similarly hold environment information at a low level of abstraction, such as air temperature or raw data from a motion sensor, or at higher levels of abstraction, such as the number and identities of nearby people, objects, and locations.
  • the model of the user's condition can additionally include information added explicitly from other sources (e.g., application programs), as well as user-specified or system-learned defaults and preference information.
  • An illustrative example of a model of a user condition is described in greater detail with respect to Figure 3.
  • the CDOS system includes a User Characterization Module, an Output Device Selector Module, and a Format Module associated with each available output device.
  • the User Characterization Module monitors the user and the user's environment in order to create a current model of the user's condition. After the User Characterization Module has created a model of the user's current condition, the Output Device Selector Module and the one or more Format Modules can then use the model to determine when and how to present output information to the user.
  • the User Characterization Module can receive a variety of types of information, and can use this information to determine the user's current condition in a variety of ways.
  • the User Characterization Module can receive user input supplied by the user to the computer system, information about the user sensed from a variety of sensor devices, information about the environment surrounding the user received from a variety of sensor devices, indications from the CDOS system about output information currently being presented to the user, stored background information about the user or about the world, and various types of information from external entities such as application programs.
  • User input information alone can provide significant information to the CDOS system about the user's current condition. For example, if the user is currently supplying input to the computer via a full-sized keyboard, it is likely that the user is engaged in little other physical activity (e.g., walking), that the user is devoting a significant amount of attention to the computer system, and that the user would see information flashed on the display.
  • the user is currently supplying input to the computer via a full-sized keyboard, it is likely that the user is engaged in little other physical activity (e.g., walking), that the user is devoting a significant amount of attention to the computer system, and that the user would see information flashed on the display.
  • va ⁇ ety of input devices with which a user can supply information to the computer system, including voice recognition devices, traditional qwerty keyboards, chording keyboards, half qwerty keyboards, dual forearm keyboards, chest mounted keyboards, handwriting recognition and digital ink devices, a mouse, a track pad, a digital stylus, a finger or glove device to capture user movement, pupil tracking devices, a gyropoint, a trackball, a voice g ⁇ d device, digital cameras (still and motion), etc
  • voice recognition devices traditional qwerty keyboards, chording keyboards, half qwerty keyboards, dual forearm keyboards, chest mounted keyboards, handwriting recognition and digital ink devices, a mouse, a track pad, a digital stylus, a finger or glove device to capture user movement, pupil tracking devices, a gyropoint, a trackball, a voice g ⁇ d device, digital cameras (still and motion), etc
  • voice recognition devices traditional qwerty keyboards, chording keyboards
  • Characte ⁇ zation Module also uses sensed information about the user For example, a va ⁇ ety of sensors can provide information about the current physiological state of the user, geographical and spatial information (e g , location and altitude), and current user activities. Some devices, such as a microphone, can provide multiple types of information.
  • the microphone can provide sensed information related to the user (e g , detecting that the user is talking, sno ⁇ ng, or typing) when not actively being used for user input
  • Other user-worn body sensors can provide a va ⁇ ety of types of information, including that from thermometers, sphygmometers, heart rate sensors, shiver response sensors, skin galvanometry sensors, eyelid blink sensors, pupil dilation detection sensors, EEG and EKG sensors, sensors to detect brow furrowing, blood sugar monitors, etc
  • sensors elsewhere in the near environment can provide information about the user, such as motion detector sensors (e g , whether the user is present and is moving), badge readers, still and video cameras (including low light, infra-red, and x-ray), remote microphones, etc
  • These sensors can be both passive (/ e , detecting information generated external to the sensor, such as a heart beat) or active ( ⁇ e , generating a signal to obtain information, such as sonar or
  • Such information typically includes information about the user that changes at most infrequently although it is possible to frequently update the stored background information to reflect changing conditions.
  • background information about the user can include demographic information (e.g., race, gender, age, religion, birthday, etc.) if it can affect how information is presented to the user.
  • User preferences either explicitly supplied or learned by the system, can also be stored as background information.
  • Information about the user's physical or mental condition which affects the type of information which the user can perceive, such as blindness, deafness, paralysis, or mental incapacitation, is also important background information that allows the CDOS system to adapt to the user's capabilities.
  • the User Characterization Module receives and uses information related to the environment surrounding the user. For example, devices such as microphones or motion sensors may be able to detect whether there are other people near the user and whether the user is interacting with those people. Sensors can also detect environmental conditions which may affect the user, such as air thermometers or geigercounters.
  • Sensors can also provide information related to a wide variety of user and environment factors including location, orientation, speed, direction, distance, and proximity to other locations (e.g., GPS and differential GPS devices, orientation tracking devices, gyroscopes, altimeters, accelerometers, anemometers, pedometers, compasses, laser or optical range finders, depth gauges, sonar, etc.).
  • locations e.g., GPS and differential GPS devices, orientation tracking devices, gyroscopes, altimeters, accelerometers, anemometers, pedometers, compasses, laser or optical range finders, depth gauges, sonar, etc.
  • Identity and informational sensors e.g., bar code readers, biometric scanners, laser scanners, OCR, badge readers, etc.
  • remote sensors e.g., home or car alarm systems, remote camera, national weather service web page, a baby monitor, traffic sensors, etc.
  • the User Characterization Module can also receive information from devices which aggregate low-level information into higher-level data constructs (e.g., face recognizers, gesture recognition systems, affective/emotion recognizers, etc.).
  • the user can also explicitly supply information about their current condition (e.g., "I have a high cognitive load and do not want to be disturbed,” “I am dealing with private family information that should not be made available to anyone outside my immediate family,” or "I am in an environment where I do not wish to disturb others around me”).
  • the CDOS system can also receive current date and time information in order to both track changes over time and to utilize information such as the user's stored schedule.
  • Previously-created models of the user's condition can also be retrieved and used as a default or to detect changing conditions.
  • the Output Device Selector Module can also supply information indicating the types of output currently being presented to the user, thus providing information about the user's current activities and cognitive load.
  • other entities e.g., application programs
  • user condition information e.g., new condition variables, whether application-specific or not, or new values for existing condition variables.
  • CDOS systems communicate between themselves, such as via a wireless medium or when cabled together. This intercommunication can occur automatically, or at the instruction of one or more of the users of the communicating CDOS systems.
  • a variety of types of information can be passed between the CDOS systems.
  • a first CDOS system receiving information from other CDOS systems can use those systems as a type of remote sensor in which information received by the User Characterization Modules of the other CDOS systems is also supplied as input to the User Characterization Module of the first CDOS system.
  • Other CDOS systems may have access to information about the surrounding environment (e.g., a digital camera) that the first CDOS system does not have.
  • CDOS systems can also act as cooperative systems in which one or more users' CDOS systems are shared with other users (e.g., making available excess computing power or the use of an output device).
  • the User Characterization Module After the User Characterization Module receives information about the user and the surrounding environment from one or more information sources, the User Characterization Module will use this information to create a current model of the user's condition. In one embodiment, the User Characterization Module merely stores the data it receives (even when it is at a low-level of abstraction) and then allows the Output Device Selector Module or application programs to directly use that stored information when making decisions related to presenting output information. In an alternate embodiment, the User Characterization Module uses received low-level data to generate higher-level representations of the user's observable activities (e.g., walking, watching a movie in a movie theater, talking to coworkers at the office, etc.).
  • the User Characterization Module uses received low-level data to generate higher-level representations of the user's observable activities (e.g., walking, watching a movie in a movie theater, talking to coworkers at the office, etc.).
  • the User Characterization Module further characterizes the user's condition with respect to condition variables that are not directly observable.
  • condition variables include the current cognitive load of the user (indicating amount of attention required for the user's current activities and thus the ability of the user to devote attention to the computer), the current degree of interruptibilitv for the user (indicating ability to safely interrupt the user), the current degree of intrusiveness of output on the environment (indicating impact of output on the surrounding environment), the user's desired scope of audience for information being output (indicating how many people should be able to perceive the information), the user's desired level of privacy for information being output (indicating the group of people who are allowed to perceive the information), and the user's desired level of solitude (indicating the user ' s current desire to avoid intrusions).
  • User condition variables can also represent abstract principles about the user and the surrounding environment, such as the user's relationship to other objects, people, or locations (e.g., being at their desk, being in their office, being near the drug store, talking to a particular person, etc.).
  • CDOS systems can supply information about user condition variables and their values to other CDOS systems, and those other CDOS systems can add the user condition variables and/or values to their model of their user condition if appropriate (e.g., ambient air temperature, or an emotional state of a CDOS system's user that is sensed by another CDOS system).
  • the values for the user condition variables can directly impact how and when output information should be presented. For example, when the cognitive load is high or the degree of interruptibility is low, output information may be presented on a tactile display using light pressure so that the presentation is minimally intrusive on the user. Alternately, the presentation of the information may be deferred if no appropriate output device is available or if interrupting the user is not warranted by low-importance or low-urgency information. When the output information is sensitive and others present are not included in the current desired level of privacy, the information may be presented on an eyeglass-mounted display, or the information may be presented via an earpiece speaker when the scope of audience or intrusiveness on the surrounding environment dictates that others not perceive the presented information.
  • the User Characterization Module may receive contradictory information related to one or more aspects of the user condition.
  • a motion sensor device may indicate that no one else is present in a room, while a speech recognizer may report that another person is present.
  • Mediation of such contradictory data can be handled in a variety of ways. For example, it may be possible to reconcile such data (e.g., the user is communicating with another person via a telephone with a loudspeaker). Alternately, the data could reflect different readings for changing conditions (e.g., ambient air temperature may have changed quickly after a window was opened).
  • the values for the variables can be stored in a variety of ways (e.g., a number on a scale of 1-100 or 0-255, a probability distribution, a value from a delimited set of possibilities, a fuzzy logic value, etc.).
  • Factors which can affect the cognitive load of a user include if the user is talking (and the volume of the speech), is talking on the phone, physical movement such as walking or driving, being stationary, being seated and stationary, ambient l.
  • a level of rest e.g., a low level due to a recent lack of sleep
  • activity such as reading e-mail or riding a bull
  • historical data e.g., user has low threshold for cognitive load while watching baseball games
  • location e.g., at home or therapist's office
  • presence and frequency of user input such as keyboard or mouse activity, presentation of output information to the user, emotional state, explicit indications from user, etc.
  • factors that can affect desired level of privacy and desired scope of audience include the identity of others near the user, the proximity of others to the user, explicit tagging of activities or information (e.g., email in my personal account is private for only me, while email in my family account is private for family members), nature of work being performed (e.g., balancing a checkbook, playing a computer game, or revising a business spreadsheet), location, historical data, explicit indications from user, etc.
  • the components of the CDOS system have a shared context as to the meaning of input information and user condition variables, including having consistency among the components generating values of the user condition variables (e.g., rules in the User Characterization Module) and those using the generated values (e.g., the Output Device Selector Module and Format Modules).
  • the components e.g., user condition variables and their values
  • other entities e.g., other CDOS systems or application programs with output information to be presented
  • sharing of the context with these other entities enables the information to be useful.
  • other systems are designed to share the same context (e.g., via a published API), while in other embodiments additional information describing the shared information can be supplied along with the shared information to establish a shared context.
  • the User Characterization Module can generate the model of the current user condition in a variety of ways.
  • the User Characterization Module includes a variety of rules such that when a test portion of the rule is currently true, then the result po ⁇ ion of the rule is activated or performed (e.g., to cause the value of a condition variable to be modified or to satisfy the test portion of other rules).
  • a rule could indicate that if the user is talking or the surrounding environment is loud, then non-auditory output is preferable to auditory output.
  • a second rule such as a rule stating that while non-auditory output is currently preferable then an eyeglass-mounted display device will be used for output.
  • a second rule could state that although non-auditory output is currently preferable, an earpiece speaker device will be used for highly sensitive information.
  • Another example of a rule is that if the user is driving an automobile at high speed in bad weather conditions, it is likely that the user has a high cognitive load and would be unable to safely devote much attention to the computer system.
  • values for some user condition variables may be calculated only periodically or only upon specific request for the value (e.g., computationally intensive variable values such as those generated by a face recognizer), even if the appropriate input information is supplied more frequently.
  • some embodiments of the CDOS system may allow the User Characterization Module to request or retrieve the appropriate input information needed to calculate one or more user condition variables, thus performing demand- driven processing.
  • An illustrative example of a User Characterization Module is described in greater detail with respect to Figure 4.
  • CDOS systems can supply to other CDOS systems various information related to generating the model of the user condition, and those other CDOS systems can use that model generation information in addition to or in place of their own model generation information. For example, if rules are being used to generate the model of the user condition, one CDOS system can supply some or all of its rules to other CDOS systems. Similarly, default and/or specialized sets of model generation information can be supplied to a CDOS system, either from other CDOS systems.
  • CDOS systems or by loading that information onto the CDOS system.
  • a default set of rules may be used by CDOS system until learning by the system adds or modifies the default rules to better model the user of the system.
  • other programs e.g., application programs
  • rules can supply rules to the CDOS system, such as rules specific to that application program.
  • sets of rules can also be supplied. For example, sets of rules may be specialized based on occupation (e.g., a nurse, a secretary, a field technician, or a firefighter), gender (e.g., a woman's rules may understand physiological symptoms related to pregnancy or other female-specific conditions), age, or any of a variety of other specialization types.
  • the Output Device Selector Modules can then use the model to determine when and how to present output information to the user.
  • the Output Device Selector Module first receives output information to be presented to the user (e.g., from an application program), and then selects an appropriate output device for presenting the information to the user. For example, if the model of the user condition indicates that auditory output is currently preferable to other forms of output and the output information can be presented audibly, then the Output Device Selector Module selects an output device that supports audible output. Alternately, the value of a desired level of privacy, desired scope of audience, or current cognitive load user condition variable may indicate that audible output is currently preferable.
  • external entities such as an application program can directly access the model of the user condition and make their own determination as to when, where and how to present output information (i.e., bypassing the Output Device Selector Module and/or the Format Modules).
  • the external entity can postpone or cancel the presentation of the output information without ever supplying the output information to the CDOS system.
  • the Format Module for the output device will then format the output information appropriately based on the model of the user condition (e.g., lowering the volume to minimize the degree of interruptibility for non-urgent information or to prevent non-employees of the business from perceiving sensitive business information, or using 3-D sound that is output as if from a sound source located at a particular location in the user's surrounding environment).
  • the Output Device Selector Module instead defers or denies the presentation of the information.
  • the Output Device Selector Module selects output devices by first characterizing each of the output devices relative to selected condition variables, such as cognitive load, desired level of privacy, desired scope of audience, and intrusiveness on the environment. For example, an eyeglass-mounted display may have a high rating for ability to present sensitive information to only the user, but may have a low rating for lack of intrusiveness on the user (particularly if the user has a high cognitive load from another visual activity).
  • an olfactory device which can output various smells may be low on the intrusiveness scale, but may be useful for presenting only limited types of output (e.g., a soothing aroma when the user has high blood pressure and a high pulse).
  • Output devices can also be characterized on the basis of the user sense (e.g., olfactory or visual) to which the output information will be presented.
  • the Output Device Selector Module selects the one or more output device which are most appropriate for the user's current condition and for the information to be output.
  • a characterization of a device relative to a condition variable is dependent on the circumstances rather than being inherent in the capabilities of a device. For example, a stereo or a television may have a high degree of privacy while only the user is in the room, but the characterization for these devices may change to a low degree of privacy when others enter the room.
  • such devices are represented with a characterization that is a range of values, with only a single value or a subset of the range selected at a given time based on the current circumstances.
  • an external entity can also supply information that describes the output information, such as the relative importance and urgency (i.e., the degree of deferability, such as time sensitivity) of the information, as well as the consequences of ignoring the information.
  • the output devices can be characterized relative to condition variables, they can also be characterized relative to such factors in the description information. For example, an eyeglass-mounted display and an earpiece speaker with adjustable volume may both be highly-rated with respect to their ability to present important information that has a high consequence of being ignored.
  • the earpiece speaker may have a wide range of ratings for these factors, however, since it is also able to present low importance information (e.g., at a low audio volume which can be easily ignored by the user if the user so chooses). Conversely, the eyeglass-mounted display may not be able to unobtrusively present visual information, and thus may have a small range of ratings for this factor.
  • the Output Device Selector Module uses the model of the user condition to determine which output device (if any) to use to present the information to the user, and a corresponding Format Module for that device determines the appropriate format with which to present the information to the user.
  • the Output Device Selector Module includes a characterization of each output device available to the CDOS system relative to the user condition variables of cognitive load, desired level of privacy, desired scope of audience, and desired level of intrusiveness on others, as well as to output information description factors of relative level of importance, deferability, and consequence of ignoring.
  • the one or more devices which best match the current user condition and the current output information will be selected, including using user preferences to select between different devices.
  • the Output Device Selector Module could determine an appropriate output device in a variety of other ways, including receiving a direct specification from the entity supplying the output information, selecting the device with the widest range of capabilities relative to the type of information to be output, etc.
  • FIG. 1 illustrates a body-mounted wearable computer 120 worn by user 1 10, with the computer suitable for executing an embodiment of the CDOS system 100
  • the user has a va ⁇ ety of body-worn input devices including a microphone 124, a handheld flat panel display 130 with character recognition capabilities, and vanous other user input devices 122.
  • the user has a va ⁇ ety of body-worn output devices that include the hand- held flat panel display, an earpiece speaker 132, an eyeglass- mounted display 134, and a tactile display 136
  • the CDOS system can also receive information from vanous body- worn user sensor devices 126 and environment sensor devices 128 As the user moves about in vanous environments, the CDOS system receives vanous input information, maintains a current model of the user condition, and presents output information to the user via approp ⁇ ate output devices
  • the user is accessible to a computer 150 (e g , in close proximity to or reachable via a long-distance communication device such as a cellular phone) which also has a va ⁇ ety of input and output devices.
  • the computer is non-portable, although the body-mounted computer of the user can similarly communicate with a vanety of other types of computers, including body-mounted computers of other users
  • the devices from which the non-portable computer can directly receive information include vanous user input devices 152 and vanous user sensor devices 156
  • the non-portable computer can output information directly to a display 160, a speaker 162, an olfactory device 164, and a pnnter 166
  • the body-mounted computer can communicate with the non-portable computer via a wireless transmission medium
  • the CDOS system can receive information from the user input devices 152 and the user sensor devices 156 after the information has been transmitted to the non-portable computer
  • the body-mounted computer may be able to directly
  • the computer systems may contain additional components or may lack some illustrated components.
  • the CDOS system could be executed on the non-portable computer, with the body-mounted computer replaced by a thin client such as a transmitter/receiver for relaying information between the body-mounted input and output devices and the nonportable computer.
  • a thin client such as a transmitter/receiver for relaying information between the body-mounted input and output devices and the nonportable computer.
  • no devices or computers may be worn by the user.
  • the body-mounted computer may be connected to one or more networks of other devices through wired or wireless communication means (e.g., wireless RF, a cellular phone or modem, infrared, physical cable, a docking station, etc.), either with or without support from other computers such as the computer 150.
  • wired or wireless communication means e.g., wireless RF, a cellular phone or modem, infrared, physical cable, a docking station, etc.
  • the body-mounted computer of a user could make use of output devices in a smart room, such as a television and stereo when the user is at home, if the body- mounted computer can transmit information to those devices via a wireless medium or if a cabled or docking mechanism is available to transmit the information.
  • kiosks or other information devices can be installed at various locations (e.g., in airports or at tourist spots) to transmit relevant information to body-mounted computers within the range of the information device.
  • a wearable defibrillator could be created such that the CDOS system monitors the user's physiological condition to determine whether output electrical energy needs to be supplied to the user, and to automatically supply such output energy when needed.
  • a breathing monitor system could monitor the user and take appropriate action if a breathing problem develops (e.g., calling 91 1 or notifying a nearby medical care provider), or a sleep-sensing system for a dnver of a vehicle could stimulate the dnver (e g , audibly or tactilely) when the dnver becomes drowsy
  • a vanety of other physiological conditions can be similarly monitored, and other specialized versions of the system can similarly be implemented (e g , an aid for a deaf person that performs speech recognition on spoken words in order to visually display the words, or a mood enhancing device that t ⁇ ggers vanous soothing environmental output in response to a user condition such as stress or anger)
  • FIG. 2 illustrates an embodiment of the body-mounted computer 120 in greater detail
  • the computer includes a memory 270, a CPU 280, and a storage device 290
  • the CDOS 100 system is executing in memory, as well as one or more distinct application programs 260
  • the information is forwarded to the User Charactenzation Module 205 of the CDOS system
  • These types of information include explicit user input to the computer, sensed user information, and sensed environment information
  • the User Charactenzation Module can also receive date and time information from the CPU or from some other source, and can retneve stored information (e g , user preferences, definitions of vanous user-defined groups, or a default model of the user condition) from the storage device
  • one or more of the application programs can optionally supply application-supplied information 265 to the User Charactenzation Module.
  • This information can include any type of user condition information to which the application program has access, such as user location or physiological state.
  • the application programs can create new user condition va ⁇ ables (e g , an indication of where the user's pupil is directed for an interactive game program), including those to be used only by that application program Similarly, a utility program could supply user condition information that is useful to a specified subset of application programs (e g , to vanous application programs from a single vendor or of a certain type)
  • the ser Charactenzation Module After the ser Charactenzation Module receives one or more of these types of information, it processes the information and creates a model of the user condition 210 which will include multiple user condition va ⁇ ables (with cunent values for some or all of the variables). After the model of the user condition has been created, the current model will be made available to the User Characterization Module to assist in the characterization of the user, such as with changes over time. The model will also be available to the Output Device Selector Module 215 to assist with presentation of output information. Moreover, the model of the user condition can additionally be stored in a permanent manner, such as on the storage device, if non-current versions of the user condition model are useful. Similarly, the User Characterization Module, Output Device Selector Module, and any Format Modules can be permanently stored before being executed in memory, and any changes made to the modules while they are executing can also be saved.
  • the Output Device Selector Module When the Output Device Selector Module receives output information to be presented, such as from one of the application programs, it uses the current model of the user condition as well as information about the available output devices to determine an appropriate output device on which to present the information to the user. In some embodiments, the Output Device Selector Module may retrieve information about the output device characteristics upon initialization, such as from the storage device. Alternately, the Output Device Selector Module could instead receive the information directly from the output devices as they are dynamically configured. The source of the output information can also supply a description of the information to assist in selecting where, when and how to present the information to the user. After an output device has been selected, the Output Device Selector Module forwards the output information as well as appropriate output information description factors and user condition variables to the Format Module for the output device. In the illustrated embodiment, Format Modules 220 through 228 correspond to the output devices as shown.
  • a Format Module When a Format Module receives the output information, it formats the output information for presentation, with the formatting based in part on the information presentation capabilities of the output device. For example, the output device may be able to output information to more than one user sense, in more than one way, and with varying degrees of amplitude or style (e.g., flashing text or enlarged icons).
  • the Output Device Selector Module selects an appropriate method of formatting the information, such as to present the information to only the appropriate audience or with the appropriate level of intrusiveness, and then sends the information to its conesponding output device for display.
  • the Output Device Selector Module will also inform the User Characterization Module when output is to take place so that the model of the user condition can be updated accordingly.
  • the Format Modules may communicate with their corresponding output devices in a variety of ways, and that the body-mounted computer in the CDOS system may contain additional components or may lack some illustrated components. For example, there may not be a one-to-one mapping between Format Modules and output devices, functionality performed by the Output Device Selector Module and Format Modules may be incorporated together, and the creation of the model of the user condition may be performed by a different system than that which uses the information to present output information. There may also be multiple User Characterization or Output Device Selector Modules, such as one User Characterization Module for each relevant high-level condition variable.
  • external entities such as the application programs could add their own User Characterization, Output Device Selector or Format Modules, or could directly access the model of the user condition in order to perform presentation of output information. Accordingly, the present invention may be practiced with other computer system configurations.
  • Figure 3 is an illustrative example of a Model of User Condition 210.
  • the model reflects the condition of user X at time 14:22 hours on the displayed date.
  • the illustrative model of the user condition includes a variety of user condition variables at different levels of abstraction, including low-level information directly from user sensors as well as higher-level abstract variables with characterized values that reflect a user's current physical and mental states. Historical and time- sensitive information can also be included, as shown by the variable illustrating the last user input performed by user X.
  • Intermediate-level variables included in the model can be calculated from low-level input information such as sensor values.
  • the speed of the user could be calculated directly by a sensor such as a pedometer, or could be calculated indirectly via information over time from a GPS sensor.
  • the Speed variable indicates that additional information can be included in the user model for each variable. In the case of the Speed variable, uncertainty about the exact value of the variable is demonstrated.
  • Other calculated condition variables include an indication that the user is located in their office, is near their desk, and that there are no other people physically nearby. These factors can be determined in a variety of ways, such as via a motion sensor device located on the desk that is tracking the user and the absence of other individuals, or by the lack of any sounds from any other people via one or more microphones.
  • Higher-level condition variables can also be calculated, such as the user's current physical activities, the current user cognitive load, the desired level of privacy, and the desired scope of audience.
  • Information from the microphone or directly from the cellular phone could indicate that the user is currently talking on their cellular phone, and the speed and motion sensor data could indicate that the user is walking. Since the user remains near his desk even though he is walking, the system can deduce that the user is pacing about his office or is walking on a treadmill (not shown).
  • the User Activity variable demonstrates that variables can have multiple values, and that information such as a degree of belief or certainty in the value for a variable can be added and used by the system.
  • the Cognitive Load variable indicates a score of 77 out of 100, thus indicating a relatively high cognitive load due to the combination of the user walking and talking on the phone. Since it is unlikely that information presented by the system will be desired to be perceptible by the person on the other end of the phone, the desired Scope Of Audience variable indicates that only the user is currently appropriate to receive output information.
  • the desired Level Of P ⁇ vacy vanable indicates that if information is presented in a manner such that the other person can receive it (e g , through an external speaker), general information about the company as well as executive-level sensitive information can be presented Note that although low-level sensors such as a motion detector may have indicated that there are no other people physicallv nearby, when it was determined that the user was talking on a phone, additional information as added to the Nearby People vanable to indicate that someone is within audio perception of the user
  • the remaining displayed portions of the user condition model indicate that user preference information and externally supplied information can be included in the user condition model
  • the Application X-Factor 1 vanable has been supplied by application X, as well as a value for the vanable
  • the value is a normal probability distnbution with a mean of 23 and a standard deviation of 3
  • previously supplied user preference information could indicate which output devices and which output formats are preferred by the user
  • the system could have automatically learned these preferences over time by observing user reactions to vanous outputs, as well as from explicit suggestions and ovemdes by the user
  • the illustrated user condition model is merely illustrative and is not intended to limit the scope of the present invention
  • the model may contain additional va ⁇ ables or may lack some illustrated va ⁇ ables, or may be represented without explicit condition va ⁇ ables at all
  • FIG 4 is an illustrative example of User Charactenzation Module 205
  • the illustrated User Charactenzation Module is for user X and it includes a vanety of IF-THEN rules
  • User condition va ⁇ ables are shown with angle brackets surrounding them, with some user condition v a ⁇ ables (e g , Speakerphone Status) not shown in the illustrative model of user condition 210 in Figure 3
  • WHILE-THEN rules are also shown, as well as an application-specific rule (i e .
  • the illustrative ser Characte ⁇ zation Module also indicates that the results portion of the rules (shown after the THEN statements) can set or modifv the alues of condition v a ⁇ ables, such as bv absolute or percentage nume ⁇ cal amounts, and can indicate degrees of be et or uncertainty in values. Groups of people are shown in square brackets (e.g., Company Executives), and asterisks are wild-card characters that can match any information.
  • receiving input related to one user condition variable can cause multiple changes to propagate through the set of rules. For example, if input is received that indicates that the user condition variable
  • the illustrated User Characterization Module is merely illustrative and is not intended to limit the scope of the present invention.
  • the model may contain additional rules, may lack some illustrated rules, or may be implemented without using rules at all.
  • the test and/or results portions of rules can be implemented as invokable functions, including those provided by external entities such as application programs.
  • FIG. 5 is an illustrative example of Output Device Selector Module 215. As is shown, the module is for user X and it maps each available output device to ratings for selected user condition variables and output information description factors. As is shown, some output devices which are available at times (e.g., pager 502. cellular telephone 504, and car radio 506) are not currently available. In addition, earpiece speaker 132 may not currently be able to receive output information if it is already in use (e.g., the user is listening to music).
  • output devices which are available at times (e.g., pager 502. cellular telephone 504, and car radio 506) are not currently available.
  • earpiece speaker 132 may not currently be able to receive output information if it is already in use (e.g., the user is listening to music).
  • new output information could preempt the current use of the earpiece speaker if necessary, or could instead share the use of the output device (e.g., outputting music to one ear and other information to the other ear if the earpiece speaker is part of headphones).
  • the various output devices are rated with single values or a range of values for each factor. While textual values are used, those skilled in the art will appreciate that nume ⁇ cal or other types of rating systems could be used.
  • ranges may illustrate the device capabilities in different situations, with the ranges being restncted in any particular situation
  • the earpiece speaker can accommodate when the user has a very low cognitive load by adjusting the volume to be slightly louder than the ambient environment Alternately, even if the user has a high cognitive load, the earpiece speaker can interrupt the user if necessary for urgent information by using very loud volumes or distinctive tones
  • the ratings can be adjusted to reflect the specific situation of this user For example, since the speaker 162 is located on the user's desk at work and other employees can frequently or always hear the speaker, the value for the desired Level Of Pnvacy may indicate that only business information be presented via the speaker Alternately, the system could present information DV sending it to the cellular telephone if the information is highly sensitive or it is important to interrupt
  • the illustrated Output Device Selector Module is merely illustrative and is not intended to limit the scope of the present invention
  • the module may contain additional user condition va ⁇ ables and output information descnption factors, may lack some illustrated user condition va ⁇ ables and output information descnption factors, or may select output devices on which to present output information in an entirelv different manner
  • some embodiments of the Output Device Selector Module may include specific logic, such as
  • Figure 6 is an exemplary flow diagram of an embodiment of the User
  • the User Charactenzation Routine ret ⁇ eves stored information related to the user, including a default model of the user condition, receives va ⁇ ou-. types of input information related to the user or the user's environment, updates the model of the user condition to reflect the new information, and pe ⁇ odically updates the model if no information has been received within a presc ⁇ bed time
  • the routine begins at step 605 where stored information for the user is ret ⁇ ev ed.
  • routine including a set of characte ⁇ zation rules to be used bv the routine
  • the routine then creates a default model of the user condition, such as directlv from stored information or by applvmg the characte ⁇ zation rules to default user information that is available
  • the routine then continues to step 610 to set a timer, and continues to step 615 to either receive input information or to receive a notification that the timer has expired
  • step 622 determines if input information was received If so, the routine continues to step 625 to determine if the information received was information input to the computer by the user If so, the routine continues to step 630 to determine if the user input indicates that the user condition should be modified, such as bv setting a user preference or explicitly changing the value of a user condition vanable. If so, the routine continues to step 635 to satisfy the user request, and then returns to step 610 If it was instead determined in step 630 that the user input was not directed to the User Characte ⁇ zation Module, the routine continues to step 640 to forward the user input information to the appropnate destination (e g , an application program).
  • the appropnate destination e g , an application program
  • step 640 or if it was determined in step 620 that the timer had expired or in step 625 that the information received was not user input, the routine continues to step 645 to execute the Characte ⁇ ze User Subroutine to update the model of the current user condition
  • step 645 the routine continues to step 652 to store the updated user condition model, and then continues to step 655 to update the characte ⁇ zation rules if necessary
  • the charactenzation rules can be updated in a va ⁇ ety of situations, such as if an external entity (e.g , an application) explicitly adds a new charactenzation rule or if the routine monitors the user's behavior and reactions in an attempt to learn more approp ⁇ ate characte ⁇ zation rules.
  • step 660 determine if there is more information to receive. If so. the routine returns to step 610, and if not, the routine ends at step 695
  • Figure 7 is an exemplary flow diagram of an embodiment of the
  • the subroutine is executed when information is received related to the user or the user's env ironment, or when a timer has expired indicating that no information has been receiv ed for a specified pe ⁇ od of time
  • the model of the user ' s current condition may need to o updated so that time-sensitive information can be updated in the model of the user condition
  • the subroutine begins at step 705 where the current model of the user condition is retneved
  • the suoroutine then continues to step 710 to retneve the current date and time
  • the subroutine determines whether information was received or if the timer has expired If the timer has expired, the subroutine continues to step 720 to examiner user condition va ⁇ ables tat represent time-sensitive information or histo ⁇ cal data and updates them if necessary
  • the subroutine then continues to step 725 to determine if the current date and time tngger any
  • step 740 the subroutine continues to step 740 to add the new charactenzation rule, determine if current v alues for any user condition vanables t ⁇ gger the rule, and if so propagates any changes from the tnggered rules through the set of rules If it was instead determined in step 735 that a new charactenzation rule is not being defined, the subroutine continues to step 745 to determine if the current input information or the current date and time tngger any rules, and if so, changes from those tnggered rules are propagated throughout the set of rules In addition to information received directly from the user, sensors, or application programs, this input information can also be a notification from the Output Device Selector that indicates output information is currently being presented to the user After steps 725, 730, 7 40, or 745 the subroutine continues to step 750 to store any changes in user condition vanables and their values, as well as the new date and time, in an updated model of the user condition The subroutine then continues to step 740 to
  • Figure 8 is an exemplary flow diagram of an embodiment of the Output
  • the Output Dev ice Selector Routine 800 receives output information to be presented to the user, ret ⁇ ev es cunent charactenzed information about the user from the model of the user condition, determines whether the output information should be presented to the user at this time and if so on which output device and in what format, and then notifies the User Characte ⁇ zation Module when output information is presented to the user
  • the routine begins at step 805 where output information to be presented to the user is recei ed or an indication that a timer has expired occurs
  • the routine continues to step 810 to determine if a timer has expired
  • step 810 When output information cannot be currently presented to the user (e g , no satisfactory output device is available or presentation to the user could be dangerous or inapprop ⁇ ate), the presentation is deferred and a timer is set to indicate when to rev lew presenting the information
  • step 815 the routine continues to step 815 to ret ⁇ eve the deferred output information for the timer, as well as any descnption information for the defened output information
  • the routine continues to step 820 where descnption information for the output information is optionally received
  • step 825 to ret ⁇ eve relevant information from the cu ⁇ ent model of the user condition
  • step 830 to determine whether to cunently present the output information to the user. In the illustrated embodiment, this determination is made using the user condition vanables of cognitive load, desired level of p ⁇ vacy, and desired scope of audience.
  • descnption information which indicates the importance and the deferability of the output information and the consequences of the user igno ⁇ ng or not receiving the output information are considered, as is any user preference information Cunent alues for these user condition vanables and descnption factors as well as whether av ailable output dev ices can support the necessary formatting oi information (e g , presenting information to the approp ⁇ ate scope of audience or at the approp ⁇ ate level of intrusiveness for the user ' s cognitive load), are thus used in the determination.
  • oi information e g , presenting information to the approp ⁇ ate scope of audience or at the approp ⁇ ate level of intrusiveness for the user ' s cognitive load
  • step 835 determines whether the presentation of the information is to be deferred or not. If the presentation is to be deferred, the routine continues to step 840 to store the output information as well as its description information, and to set a timer for the information at which time the presentation of the output information will be reconsidered. If it is instead determined in step 835 that the information presentation is not to be deferred, the routine continues to step 845 where an available output device is selected. In the illustrated embodiment, the output device whose information display capabilities and ratings best match the user condition variables and description information factors of interest is chosen. The routine then continues to step 850 to execute the Format .And Present Output Information Subroutine, and then continues to step 855 to notify the User Characterization Module of the presentation of the output information.
  • step 840 or step 855 the routine continues to step 860 to determine if there are currently timers set or there is more output information to be received. If so, the routine returns to step 805, and if not the routine ends at step 895.
  • Figure 9 is an exemplary flow diagram of an embodiment of the Format
  • the subroutine receives output information to be presented and its description information, receives relevant user condition variables and user preference information, selects a user sense to which the output information will be presented (if the output device supports more than one). selects approp ⁇ ate formatting with which to present the output information, and presents the output information to the user.
  • the subroutine begins at step 905 where output information is received, as well as the descnption information factors, user condition variables, and relevant user preference information.
  • the subroutine continues at step 910 to select a user sense that is supported by the selected output device.
  • the subroutine selects formatting for the output information that is approp ⁇ ate for the user condition va ⁇ ables.
  • step 920 the subroutine continues to step 920 to present the output information to the user with the selected formatting. If the Scope Of Audience and Level of Pnvacy user condition vanables indicate that the information can be presented to other people cunently present and the selected output device supports such presentation, the information will also be presented to these other people. After step 920, the subroutine continues to step 995 and returns.
  • an output device and the formatting of the output information for that device can be performed in a va ⁇ ety of ways.
  • other user condition va ⁇ ables and descnption information factors could be used, or the selection could be made without resort to such information.
  • the user could explicitly indicate the output device and formatting desired for some or all pieces of output information (e g , in response to an notification from the system), or another er" ty (e g , an application program supplying the output information) could explicitly designate the output device and/or formatting.

Abstract

A system for controlling presentation of information to a user based on the user's current condition. In particular, the system monitors the user and the user's environment, and creates and maintains an updated model of the current condition of the user. The user condition can include a variety of condition variables, including abstract concepts such as the user's current cognitive load, desired level of privacy for output information, and desired scope of audience for output information. Upon receiving output information to be presented to the user (e.g., from an application program), the system determines an appropriate output device and an appropriate format with which to present the information to the user, and then presents the output information. The system can also receive description information about the output information that describes relevant factors for determining when and how to present the output information (e.g., the importance and urgency of the output information, the consequences of the user not receiving or ignoring the output information, etc.). Some versions of the system execute on a wearable computer having a variety of available output display devices.

Description

METHOD AND SYSTEM FOR CONTROLLING PRESENTATION OF
INFORMATION TO A USER
BASED ON THE USER'S CONDITION
TECHNICAL FIELD The present invention relates generally to computer program user interfaces, and more particularly to presenting information to a user based on the user's current condition
BACKGROUND OF THE INVENTION
As computers become increasingly powerful and ubiquitous, users increasingly use their computers for a broad vaπety of tasks. For example, in addition to traditional activities such as running word processing and database applications, users increasingly use computers as an integral part of their daily lives. Programs to schedule activities, generate reminders, and provide rapid communication capabilities are becoming increasingly popular Moreover, computers are increasingly present during virtually all of a person's daily activities. For example, hand-held computer organizers (e.g., PDAs) are increasingly common, and communication devices such as portable phones are increasingly incorporating computer capabilities Thus, users may be presented with output information from one or more computers at any time.
While advances in hardware make computers increasingly ubiquitous, traditional computer programs are not typically designed to efficiently present information to users in a wide vaπety of environments. For example, most computer programs are designed with a prototypical user being seated at a stationary computer with a large display device, and with the user devoting full attention to the display. In that environment, the computer can safely present information to the user at any time, with minimal πsk that the user will fail to perceive the information or that the information will disturb the user in a dangerous manner (e g , by startling the user while they are using power machinery or by blocking their vision while they are moving with information sent to a head-mounted display) However, in many other environments these assumptions about the prototypical user are not true, and users thus may not perceive output information (e g , failing to notice an icon or message on a hand-held display dev ice when it is holstered, or failing to hear audio information when in a noisy environment or when intensely concentrating) Similarly, some user activities may have a low degree of lnterruptibihty (t e , ability to safely interrupt the user) such that the user would prefer that the presentation of low-importance or of all information be deferred, or that information be presented in a non-intrusive manner
In addition to assuming that a user is devoting full attention to the display, current computer programs typically assume that only the user is devoting attention to the computer system Thus, current computer programs are not concerned with factors related to the user's en ironment, such as whe-aer other people around the user are disturbed by information being presented or whether sensitive information is inadvertently made available to others Instead, current computer programs typically assume that the user will request output information only at appropπate times and that the user will control whether others are pπvy to output information (e g , by oπentmg the display accordingly or adjusting speaker volumes)
However, as computers are increasingly present with users and are designed to present output information other than at a user's immediate request (e g , reminding the user of an upcoming appointment), computer programs are increasingly likely to present information in a manner that interrupts the user (and may be bothersome or dangerous), that may not be perceived by the user even if highly important and urgent, that may disturb others around the user, and that may inadvertently reveal sensitive information to others
A growing trend of using wearable computers will only exacerbate this problem Such wearable computers are designed to act as constant companions and intelligent assistants to a user, thus being available to receive input from the user at any time and to present output information to the user at any time Wearable computers are typically strapped to the user's body or mounted in a holster, and may include a vaπety of both input and output devices The close association of wearable computers to their users results in the wearable computer interacting with the user in virtually any social or business situation, and thus the likelihood of inappropπate output behavior increases
SUMMARY OF THE INVENTION
Some embodiments of the present invention provide a method and system for controlling presentation of information t a user based on the user's current condition In particular, the system monitors the user and the user's environment, and creates and maintains an updated model of the current condition of the user The user condition can include a vaπety of condition vaπables, including abstract concepts such as the user's current cognitive load, desired level of pπvacy for output information, and desired scope of audience for output information The user condition can also include condition vaπables indicating physical charactenstics (e g , deafness) and physically observable charactenstics (e g , movement or proximity to another object) of the user Upon receiving output information to be presented to the user (e g , from an application program), the system determines an appropπate output device and an appropπate format with which to present the information to the user, and then presents the output information In some embodiments, the system also receives descπption information about the output information that descπbes relevant factors for determining when and how to present the output information (e g , the importance and urgency of the output information, the consequences of the user not receiving or ignoπng the output information, etc.). The system executes in some embodiments on a wearable computer having a vaπety of available output display devices.
In one embodiment, the system presents output information to a user by first receiving information about a modeled characteπstic of the user which may include a modeled preference of the user for receiving sensitive information, a modeled indication of a current degree of interruptibilitv of the user, or a modeled preference of the user for an amount of people to perceive information presented by the computer The system then selects an output device capable of presenting the output information in accordance with the modeled characteπstic, and presents the output information on the selected output device in accordance with the modeled characteπstic In an alternate embodiment, the system presents information to a user on one of multiple available output devices The system monitors the user to collect information about a current state of the user, and then models a current user condition based on the collected information by determining a current level of pπvacy desired by the user that indicates a group of people allowed to perceive information presented by the computer, by determining a current scope of audience desired by the user that indicates how many people are intended to perceive information presented by the computer, and by determining a current cognitive load of the user that indicates ability of the user to devote attention to the computer The system then receives output information to be presented to the user, and presents the output information in a manner consistent with the modeled current user condition by selecting one of the output devices such that information presentation capabilities of the selected output device support the determined current desired level of pπvacy, the determined current desired scope of audience, and the determined current cognitive load, and by presenting the output information to the user on the selected output device, so that the presentation of information by the system satisfies the modeled current user condition
BRIEF DESCRIPTION OF THE DRAWINGS
Figure 1 illustrates a user weaπng a body-mounted computer executing an embodiment of the Condition-Dependent Output Supplier (CDOS) system of the present invention
Figure 2 is a block diagram illustrating the contents and information flow of an embodiment of the CDOS system
Figure 3 is an illustrative example of a model of a current user condition
Figure 4 is an illustrative example of a User Characteπzation Module Figure 5 is an illustrative example of an Output Device Selector Module
Figure 6 is an exemplary flow diagram of an embodiment of the User Characteπzation routine
Figure 7 is an exemplary flow diagram of an embodiment of the Characteπze User subroutine Figure 8 is an exemplary flow diagram of an embodiment of the Output Device Selector routine.
Figure 9 is an exemplary flow diagram of an embodiment of the Format And Present Output Information subroutine.
DETAILED DESCRIPTION OF THE INVENTION
A embodiment of the present invention provides a method and system for controlling presentation of information to a user based on the user's current condition. In particular, the Condition-Dependent Output Supplier (CDOS) system monitors the user and the user's environment, and creates and maintains an updated model of the current condition of the user. Upon receiving output information to be presented to the user (e.g., from an application program), the CDOS system determines an appropriate output device and an appropriate format with which to present the information to the user, and then presents the output information. In some embodiments, the CDOS system also receives description information about the output information that describes relevant factors for determining when and how to present the output information (e.g., the impoπance and urgency of the output information, the consequences of the user not receiving or ignoring the output information, etc.).
In one embodiment, the model of the user's current condition includes a variety of condition variables that represent information about the user and the user's environment at varying levels of abstraction. For example, information about the user at a low level of abstraction can include raw physiological data (e.g., heart rate and EKG) and geographic information (e.g., location and speed), while higher levels of abstraction may attempt to characterize or predict the user's physical activity (e.g., jogging or talking on a phone), emotional state (e.g., angry or puzzled), desired output behavior for different types of information (e.g., to present private family information so that it is perceivable only to myself and my family members), and cognitive load (i.e., the amount of attention required for the user's current activities). Background information which changes rarely or not at all can also be included, such as the user's age, gender and visual acuity. The model can similarly hold environment information at a low level of abstraction, such as air temperature or raw data from a motion sensor, or at higher levels of abstraction, such as the number and identities of nearby people, objects, and locations. The model of the user's condition can additionally include information added explicitly from other sources (e.g., application programs), as well as user-specified or system-learned defaults and preference information. An illustrative example of a model of a user condition is described in greater detail with respect to Figure 3.
The CDOS system includes a User Characterization Module, an Output Device Selector Module, and a Format Module associated with each available output device. The User Characterization Module monitors the user and the user's environment in order to create a current model of the user's condition. After the User Characterization Module has created a model of the user's current condition, the Output Device Selector Module and the one or more Format Modules can then use the model to determine when and how to present output information to the user. The User Characterization Module can receive a variety of types of information, and can use this information to determine the user's current condition in a variety of ways. For example, the User Characterization Module can receive user input supplied by the user to the computer system, information about the user sensed from a variety of sensor devices, information about the environment surrounding the user received from a variety of sensor devices, indications from the CDOS system about output information currently being presented to the user, stored background information about the user or about the world, and various types of information from external entities such as application programs.
User input information alone can provide significant information to the CDOS system about the user's current condition. For example, if the user is currently supplying input to the computer via a full-sized keyboard, it is likely that the user is engaged in little other physical activity (e.g., walking), that the user is devoting a significant amount of attention to the computer system, and that the user would see information flashed on the display. If the user is instead generating user input audibly (e.g., through a head-mounted microphone), that fact may provide less user condition information to the CDOS system since the user can supply such audio information while engaged in a vaπety of types of physical activity Those skilled in the art will appreciate that there are a wide vaπety of input devices with which a user can supply information to the computer system, including voice recognition devices, traditional qwerty keyboards, chording keyboards, half qwerty keyboards, dual forearm keyboards, chest mounted keyboards, handwriting recognition and digital ink devices, a mouse, a track pad, a digital stylus, a finger or glove device to capture user movement, pupil tracking devices, a gyropoint, a trackball, a voice gπd device, digital cameras (still and motion), etc In addition to the information received via user input, the User
Characteπzation Module also uses sensed information about the user For example, a vaπety of sensors can provide information about the current physiological state of the user, geographical and spatial information (e g , location and altitude), and current user activities. Some devices, such as a microphone, can provide multiple types of information. For example, if a microphone is available, the microphone can provide sensed information related to the user (e g , detecting that the user is talking, snoπng, or typing) when not actively being used for user input Other user-worn body sensors can provide a vaπety of types of information, including that from thermometers, sphygmometers, heart rate sensors, shiver response sensors, skin galvanometry sensors, eyelid blink sensors, pupil dilation detection sensors, EEG and EKG sensors, sensors to detect brow furrowing, blood sugar monitors, etc In addition, sensors elsewhere in the near environment can provide information about the user, such as motion detector sensors (e g , whether the user is present and is moving), badge readers, still and video cameras (including low light, infra-red, and x-ray), remote microphones, etc These sensors can be both passive (/ e , detecting information generated external to the sensor, such as a heart beat) or active (ι e , generating a signal to obtain information, such as sonar or x-rays)
Stored background information about the user can also be supplied to the
User Characteπzation Module Such information typically includes information about the user that changes at most infrequently although it is possible to frequently update the stored background information to reflect changing conditions. For example, background information about the user can include demographic information (e.g., race, gender, age, religion, birthday, etc.) if it can affect how information is presented to the user. User preferences, either explicitly supplied or learned by the system, can also be stored as background information. Information about the user's physical or mental condition which affects the type of information which the user can perceive, such as blindness, deafness, paralysis, or mental incapacitation, is also important background information that allows the CDOS system to adapt to the user's capabilities.
In addition to information related directly to the user, the User Characterization Module also receives and uses information related to the environment surrounding the user. For example, devices such as microphones or motion sensors may be able to detect whether there are other people near the user and whether the user is interacting with those people. Sensors can also detect environmental conditions which may affect the user, such as air thermometers or geigercounters. Sensors, either body-mounted or remote, can also provide information related to a wide variety of user and environment factors including location, orientation, speed, direction, distance, and proximity to other locations (e.g., GPS and differential GPS devices, orientation tracking devices, gyroscopes, altimeters, accelerometers, anemometers, pedometers, compasses, laser or optical range finders, depth gauges, sonar, etc.). Identity and informational sensors (e.g., bar code readers, biometric scanners, laser scanners, OCR, badge readers, etc.) and remote sensors (e.g., home or car alarm systems, remote camera, national weather service web page, a baby monitor, traffic sensors, etc.) can also provide relevant environment information.
In addition to receiving information directly from low-level sensors, the User Characterization Module can also receive information from devices which aggregate low-level information into higher-level data constructs (e.g., face recognizers, gesture recognition systems, affective/emotion recognizers, etc.). The user can also explicitly supply information about their current condition (e.g., "I have a high cognitive load and do not want to be disturbed," "I am dealing with private family information that should not be made available to anyone outside my immediate family," or "I am in an environment where I do not wish to disturb others around me"). The CDOS system can also receive current date and time information in order to both track changes over time and to utilize information such as the user's stored schedule. Previously-created models of the user's condition can also be retrieved and used as a default or to detect changing conditions. The Output Device Selector Module can also supply information indicating the types of output currently being presented to the user, thus providing information about the user's current activities and cognitive load. Finally, other entities (e.g., application programs) can directly provide user condition information (e.g., new condition variables, whether application-specific or not, or new values for existing condition variables).
In some embodiments, CDOS systems communicate between themselves, such as via a wireless medium or when cabled together. This intercommunication can occur automatically, or at the instruction of one or more of the users of the communicating CDOS systems. When multiple CDOS systems communicate, a variety of types of information can be passed between the CDOS systems. For example, a first CDOS system receiving information from other CDOS systems can use those systems as a type of remote sensor in which information received by the User Characterization Modules of the other CDOS systems is also supplied as input to the User Characterization Module of the first CDOS system. Other CDOS systems may have access to information about the surrounding environment (e.g., a digital camera) that the first CDOS system does not have. Alternately, information about the users of the CDOS systems can be exchanged to facilitate further communication between the CDOS systems or between the users (e.g., notifying one user that another user has a high cognitive load and does not wish to be disturbed). Multiple CDOS systems can also act as cooperative systems in which one or more users' CDOS systems are shared with other users (e.g., making available excess computing power or the use of an output device).
After the User Characterization Module receives information about the user and the surrounding environment from one or more information sources, the User Characterization Module will use this information to create a current model of the user's condition. In one embodiment, the User Characterization Module merely stores the data it receives (even when it is at a low-level of abstraction) and then allows the Output Device Selector Module or application programs to directly use that stored information when making decisions related to presenting output information. In an alternate embodiment, the User Characterization Module uses received low-level data to generate higher-level representations of the user's observable activities (e.g., walking, watching a movie in a movie theater, talking to coworkers at the office, etc.).
In yet another embodiment, the User Characterization Module further characterizes the user's condition with respect to condition variables that are not directly observable. Such condition variables include the current cognitive load of the user (indicating amount of attention required for the user's current activities and thus the ability of the user to devote attention to the computer), the current degree of interruptibilitv for the user (indicating ability to safely interrupt the user), the current degree of intrusiveness of output on the environment (indicating impact of output on the surrounding environment), the user's desired scope of audience for information being output (indicating how many people should be able to perceive the information), the user's desired level of privacy for information being output (indicating the group of people who are allowed to perceive the information), and the user's desired level of solitude (indicating the user's current desire to avoid intrusions). User condition variables can also represent abstract principles about the user and the surrounding environment, such as the user's relationship to other objects, people, or locations (e.g., being at their desk, being in their office, being near the drug store, talking to a particular person, etc.). In some embodiments, CDOS systems can supply information about user condition variables and their values to other CDOS systems, and those other CDOS systems can add the user condition variables and/or values to their model of their user condition if appropriate (e.g., ambient air temperature, or an emotional state of a CDOS system's user that is sensed by another CDOS system).
The values for the user condition variables can directly impact how and when output information should be presented. For example, when the cognitive load is high or the degree of interruptibility is low, output information may be presented on a tactile display using light pressure so that the presentation is minimally intrusive on the user. Alternately, the presentation of the information may be deferred if no appropriate output device is available or if interrupting the user is not warranted by low-importance or low-urgency information. When the output information is sensitive and others present are not included in the current desired level of privacy, the information may be presented on an eyeglass-mounted display, or the information may be presented via an earpiece speaker when the scope of audience or intrusiveness on the surrounding environment dictates that others not perceive the presented information. Finally, if the user's desired level of solitude indicates that the user does not want to receive output information (e.g., while asleep, in the bathroom, involved in an intimate activity, etc.), presentation of all output information or of all but highly urgent and important output information may be deferred.
Those skilled in the art will appreciate that the User Characterization Module may receive contradictory information related to one or more aspects of the user condition. For example, a motion sensor device may indicate that no one else is present in a room, while a speech recognizer may report that another person is present. Mediation of such contradictory data can be handled in a variety of ways. For example, it may be possible to reconcile such data (e.g., the user is communicating with another person via a telephone with a loudspeaker). Alternately, the data could reflect different readings for changing conditions (e.g., ambient air temperature may have changed quickly after a window was opened). Alternately, when data truly conflicts, it may be impossible to reach a conclusion about a user condition variable, or the value of the variable may be represented as having varying degrees of uncertainty or belief. Both particular information sources (e.g., sensors) and particular pieces of input information can be categorized as to their quality and reliability to assist with mediation or to better model the user condition. In addition, input information can be time-stamped and otherwise identified to assist the User Characterization Module.
Those skilled in the art will also appreciate that a variety of factors can influence the determination of values for each of the condition variables, and that the values for the variables can be stored in a variety of ways (e.g., a number on a scale of 1-100 or 0-255, a probability distribution, a value from a delimited set of possibilities, a fuzzy logic value, etc.). Factors which can affect the cognitive load of a user include if the user is talking (and the volume of the speech), is talking on the phone, physical movement such as walking or driving, being stationary, being seated and stationary, ambient l. -..-t and sound, stress and hunger levels, a level of rest (e.g., a low level due to a recent lack of sleep), activity such as reading e-mail or riding a bull, historical data (e.g., user has low threshold for cognitive load while watching baseball games), a physical or mental disability, location (e.g., at home or therapist's office), presence and frequency of user input such as keyboard or mouse activity, presentation of output information to the user, emotional state, explicit indications from user, etc. Similarly, factors that can affect desired level of privacy and desired scope of audience include the identity of others near the user, the proximity of others to the user, explicit tagging of activities or information (e.g., email in my personal account is private for only me, while email in my family account is private for family members), nature of work being performed (e.g., balancing a checkbook, playing a computer game, or revising a business spreadsheet), location, historical data, explicit indications from user, etc.
Those skilled in the art will appreciate that when information is being processed and shared between multiple systems, it is necessary for a context to be shared so that a semantic understanding of what is represented by information can be conveyed. For example, merely reporting data for air temperature as being 50 is insufficient. Not only is it unclear what scale is being used (e.g., Fahrenheit or Celsius), it is also unclear exactly what information is being represented (e.g., the air surrounding the user inside a heated room, or the outside air). Thus, the components of the CDOS system have a shared context as to the meaning of input information and user condition variables, including having consistency among the components generating values of the user condition variables (e.g., rules in the User Characterization Module) and those using the generated values (e.g., the Output Device Selector Module and Format Modules). In addition, when information from the CDOS system (e.g., user condition variables and their values) is shared with other entities (e.g., other CDOS systems or application programs with output information to be presented), sharing of the context with these other entities enables the information to be useful. In some embodiments, other systems are designed to share the same context (e.g., via a published API), while in other embodiments additional information describing the shared information can be supplied along with the shared information to establish a shared context.
In addition to there being a wide variety of factors which can affect various condition variables, the User Characterization Module can generate the model of the current user condition in a variety of ways. In one embodiment, the User Characterization Module includes a variety of rules such that when a test portion of the rule is currently true, then the result poπion of the rule is activated or performed (e.g., to cause the value of a condition variable to be modified or to satisfy the test portion of other rules). For example, a rule could indicate that if the user is talking or the surrounding environment is loud, then non-auditory output is preferable to auditory output. When this first rule was satisfied, the result could trigger the satisfaction of a second rule, such as a rule stating that while non-auditory output is currently preferable then an eyeglass-mounted display device will be used for output. Alternately, a second rule could state that although non-auditory output is currently preferable, an earpiece speaker device will be used for highly sensitive information. Another example of a rule is that if the user is driving an automobile at high speed in bad weather conditions, it is likely that the user has a high cognitive load and would be unable to safely devote much attention to the computer system.
Those skilled in the art will appreciate that there are a variety of techniques for combining different types of input information and processing it to generating output information, including look-up tables, neural networks, expert systems, genetic algorithms, probabilistic belief networks, etc. In addition, values for some user condition variables may be calculated only periodically or only upon specific request for the value (e.g., computationally intensive variable values such as those generated by a face recognizer), even if the appropriate input information is supplied more frequently. Conversely, some embodiments of the CDOS system may allow the User Characterization Module to request or retrieve the appropriate input information needed to calculate one or more user condition variables, thus performing demand- driven processing. An illustrative example of a User Characterization Module is described in greater detail with respect to Figure 4.
In some embodiments, CDOS systems can supply to other CDOS systems various information related to generating the model of the user condition, and those other CDOS systems can use that model generation information in addition to or in place of their own model generation information. For example, if rules are being used to generate the model of the user condition, one CDOS system can supply some or all of its rules to other CDOS systems. Similarly, default and/or specialized sets of model generation information can be supplied to a CDOS system, either from other
CDOS systems or by loading that information onto the CDOS system. A default set of rules may be used by CDOS system until learning by the system adds or modifies the default rules to better model the user of the system. Similarly, other programs (e.g., application programs) can supply rules to the CDOS system, such as rules specific to that application program. Various specialized sets of rules can also be supplied. For example, sets of rules may be specialized based on occupation (e.g., a nurse, a secretary, a field technician, or a firefighter), gender (e.g., a woman's rules may understand physiological symptoms related to pregnancy or other female-specific conditions), age, or any of a variety of other specialization types. After the User Characterization Module has created a model of the user's current condition, the Output Device Selector Module and the one or more Format
Modules can then use the model to determine when and how to present output information to the user. The Output Device Selector Module first receives output information to be presented to the user (e.g., from an application program), and then selects an appropriate output device for presenting the information to the user. For example, if the model of the user condition indicates that auditory output is currently preferable to other forms of output and the output information can be presented audibly, then the Output Device Selector Module selects an output device that supports audible output. Alternately, the value of a desired level of privacy, desired scope of audience, or current cognitive load user condition variable may indicate that audible output is currently preferable.
While information to be presented to the user will often be generated by an entity outside the CDOS system, the CDOS system may also generate information to be presented to the user (e.g., an indication of low battery power, or of an error when adding a rule to the User Characterization Module). In addition, in some embodiments external entities such as an application program can directly access the model of the user condition and make their own determination as to when, where and how to present output information (i.e., bypassing the Output Device Selector Module and/or the Format Modules). Thus, if the modeled user condition indicates that particular output information should not currently be presented to a user, the external entity can postpone or cancel the presentation of the output information without ever supplying the output information to the CDOS system. It may also be possible to configure the CDOS system to automatically notify the external entities of the values of one or more user condition variables, such as by pushing that information to the external entities when changes occur in the values or by periodically notifying the external entities of the current values.
If the selected output device supports different ways to audibly present information (e.g., different audio levels or a choice of speakers) or the ability to output information to multiple senses, the Format Module for the output device will then format the output information appropriately based on the model of the user condition (e.g., lowering the volume to minimize the degree of interruptibility for non-urgent information or to prevent non-employees of the business from perceiving sensitive business information, or using 3-D sound that is output as if from a sound source located at a particular location in the user's surrounding environment). Alternately, if the current model of the user condition indicates that output to the user is currently inadvisable (e.g., the user has a high cognitive load, is asleep, or sensitive information cannot be safely presented in the current environment), the Output Device Selector Module instead defers or denies the presentation of the information. In one embodiment, the Output Device Selector Module selects output devices by first characterizing each of the output devices relative to selected condition variables, such as cognitive load, desired level of privacy, desired scope of audience, and intrusiveness on the environment. For example, an eyeglass-mounted display may have a high rating for ability to present sensitive information to only the user, but may have a low rating for lack of intrusiveness on the user (particularly if the user has a high cognitive load from another visual activity). Similarly, an olfactory device which can output various smells may be low on the intrusiveness scale, but may be useful for presenting only limited types of output (e.g., a soothing aroma when the user has high blood pressure and a high pulse). Output devices can also be characterized on the basis of the user sense (e.g., olfactory or visual) to which the output information will be presented.
After the output devices have been characterized on the basis of the condition variables, the Output Device Selector Module then selects the one or more output device which are most appropriate for the user's current condition and for the information to be output. In some situations, a characterization of a device relative to a condition variable is dependent on the circumstances rather than being inherent in the capabilities of a device. For example, a stereo or a television may have a high degree of privacy while only the user is in the room, but the characterization for these devices may change to a low degree of privacy when others enter the room. In some embodiments, such devices are represented with a characterization that is a range of values, with only a single value or a subset of the range selected at a given time based on the current circumstances.
In addition to supplying the output information to be presented, an external entity can also supply information that describes the output information, such as the relative importance and urgency (i.e., the degree of deferability, such as time sensitivity) of the information, as well as the consequences of ignoring the information. In the same manner that the output devices can be characterized relative to condition variables, they can also be characterized relative to such factors in the description information. For example, an eyeglass-mounted display and an earpiece speaker with adjustable volume may both be highly-rated with respect to their ability to present important information that has a high consequence of being ignored. The earpiece speaker may have a wide range of ratings for these factors, however, since it is also able to present low importance information (e.g., at a low audio volume which can be easily ignored by the user if the user so chooses). Conversely, the eyeglass-mounted display may not be able to unobtrusively present visual information, and thus may have a small range of ratings for this factor. Thus, after the Output Device Selector Module receives the information to be output and optionally receives a description of the information, the Output Device Selector Module then uses the model of the user condition to determine which output device (if any) to use to present the information to the user, and a corresponding Format Module for that device determines the appropriate format with which to present the information to the user.
In one embodiment, the Output Device Selector Module includes a characterization of each output device available to the CDOS system relative to the user condition variables of cognitive load, desired level of privacy, desired scope of audience, and desired level of intrusiveness on others, as well as to output information description factors of relative level of importance, deferability, and consequence of ignoring. The one or more devices which best match the current user condition and the current output information will be selected, including using user preferences to select between different devices. Those skilled in the art will appreciate that the Output Device Selector Module could determine an appropriate output device in a variety of other ways, including receiving a direct specification from the entity supplying the output information, selecting the device with the widest range of capabilities relative to the type of information to be output, etc. In addition, a defined API (application program interface) can be designed between external entities such as application programs and the CDOS system. The defined API will allow application programs to supply information to User Characterization Modules, extract and add information to the model of the user condition, and supply output information and description information to Output Device Selector Modules. An illustrative example of an Output Device Selector Module is described in greater detail with respect to Figure 5. Figure 1 illustrates a body-mounted wearable computer 120 worn by user 1 10, with the computer suitable for executing an embodiment of the CDOS system 100 The user has a vaπety of body-worn input devices including a microphone 124, a handheld flat panel display 130 with character recognition capabilities, and vanous other user input devices 122. Similarly, the user has a vaπety of body-worn output devices that include the hand- held flat panel display, an earpiece speaker 132, an eyeglass- mounted display 134, and a tactile display 136 In addition to the vanous body-wom user input devices, the CDOS system can also receive information from vanous body- worn user sensor devices 126 and environment sensor devices 128 As the user moves about in vanous environments, the CDOS system receives vanous input information, maintains a current model of the user condition, and presents output information to the user via appropπate output devices
In the current environment, the user is accessible to a computer 150 (e g , in close proximity to or reachable via a long-distance communication device such as a cellular phone) which also has a vaπety of input and output devices. In the illustrated embodiment the computer is non-portable, although the body-mounted computer of the user can similarly communicate with a vanety of other types of computers, including body-mounted computers of other users The devices from which the non-portable computer can directly receive information include vanous user input devices 152 and vanous user sensor devices 156 The non-portable computer can output information directly to a display 160, a speaker 162, an olfactory device 164, and a pnnter 166 In the illustrated embodiment, the body-mounted computer can communicate with the non-portable computer via a wireless transmission medium In this manner, the CDOS system can receive information from the user input devices 152 and the user sensor devices 156 after the information has been transmitted to the non-portable computer Alternately, the body-mounted computer may be able to directly communicate with the user input devices 152 and the user sensor devices 156, as well as with other vanous remote env ironment sensor devices 158, without the intervention of the non-portable computer 150 Similarly, the body-mounted computer may be able to supply output information to the display 160, the speaker 162, the olfactory device 164, and the printer 166, either directly or via the non-portable computer, and directly to the telephone 168. As the user moves out of range of the remote input and output devices, the CDOS system will be updated to reflect that the remote output devices are not currently available to receive output. Those skilled in the art will appreciate that computer systems 120 and
150, as well as their various input and output devices, are merely illustrative and are not intended to limit the scope of the present invention. The computer systems may contain additional components or may lack some illustrated components. For example, it is possible that the CDOS system could be executed on the non-portable computer, with the body-mounted computer replaced by a thin client such as a transmitter/receiver for relaying information between the body-mounted input and output devices and the nonportable computer. Alternately, no devices or computers may be worn by the user.
In addition, the body-mounted computer may be connected to one or more networks of other devices through wired or wireless communication means (e.g., wireless RF, a cellular phone or modem, infrared, physical cable, a docking station, etc.), either with or without support from other computers such as the computer 150. For example, the body-mounted computer of a user could make use of output devices in a smart room, such as a television and stereo when the user is at home, if the body- mounted computer can transmit information to those devices via a wireless medium or if a cabled or docking mechanism is available to transmit the information. Alternately, kiosks or other information devices can be installed at various locations (e.g., in airports or at tourist spots) to transmit relevant information to body-mounted computers within the range of the information device.
Those skilled in the art will also appreciate that specialized versions of the body-mounted computer and CDOS system can be created for a variety of purposes. For example, a wearable defibrillator could be created such that the CDOS system monitors the user's physiological condition to determine whether output electrical energy needs to be supplied to the user, and to automatically supply such output energy when needed. Alternately, a breathing monitor system could monitor the user and take appropriate action if a breathing problem develops (e.g., calling 91 1 or notifying a nearby medical care provider), or a sleep-sensing system for a dnver of a vehicle could stimulate the dnver (e g , audibly or tactilely) when the dnver becomes drowsy A vanety of other physiological conditions can be similarly monitored, and other specialized versions of the system can similarly be implemented (e g , an aid for a deaf person that performs speech recognition on spoken words in order to visually display the words, or a mood enhancing device that tπggers vanous soothing environmental output in response to a user condition such as stress or anger)
Figure 2 illustrates an embodiment of the body-mounted computer 120 in greater detail The computer includes a memory 270, a CPU 280, and a storage device 290 The CDOS 100 system is executing in memory, as well as one or more distinct application programs 260 As the body-mounted computer receives vanous input information, the information is forwarded to the User Charactenzation Module 205 of the CDOS system These types of information include explicit user input to the computer, sensed user information, and sensed environment information The User Charactenzation Module can also receive date and time information from the CPU or from some other source, and can retneve stored information (e g , user preferences, definitions of vanous user-defined groups, or a default model of the user condition) from the storage device It is also possible for one or more of the application programs to optionally supply application-supplied information 265 to the User Charactenzation Module. This information can include any type of user condition information to which the application program has access, such as user location or physiological state. In addition, the application programs can create new user condition vaπables (e g , an indication of where the user's pupil is directed for an interactive game program), including those to be used only by that application program Similarly, a utility program could supply user condition information that is useful to a specified subset of application programs (e g , to vanous application programs from a single vendor or of a certain type)
After the ser Charactenzation Module receives one or more of these types of information, it processes the information and creates a model of the user condition 210 which will include multiple user condition vaπables (with cunent values for some or all of the variables). After the model of the user condition has been created, the current model will be made available to the User Characterization Module to assist in the characterization of the user, such as with changes over time. The model will also be available to the Output Device Selector Module 215 to assist with presentation of output information. Moreover, the model of the user condition can additionally be stored in a permanent manner, such as on the storage device, if non-current versions of the user condition model are useful. Similarly, the User Characterization Module, Output Device Selector Module, and any Format Modules can be permanently stored before being executed in memory, and any changes made to the modules while they are executing can also be saved.
When the Output Device Selector Module receives output information to be presented, such as from one of the application programs, it uses the current model of the user condition as well as information about the available output devices to determine an appropriate output device on which to present the information to the user. In some embodiments, the Output Device Selector Module may retrieve information about the output device characteristics upon initialization, such as from the storage device. Alternately, the Output Device Selector Module could instead receive the information directly from the output devices as they are dynamically configured. The source of the output information can also supply a description of the information to assist in selecting where, when and how to present the information to the user. After an output device has been selected, the Output Device Selector Module forwards the output information as well as appropriate output information description factors and user condition variables to the Format Module for the output device. In the illustrated embodiment, Format Modules 220 through 228 correspond to the output devices as shown.
When a Format Module receives the output information, it formats the output information for presentation, with the formatting based in part on the information presentation capabilities of the output device. For example, the output device may be able to output information to more than one user sense, in more than one way, and with varying degrees of amplitude or style (e.g., flashing text or enlarged icons). The Format ??
Module selects an appropriate method of formatting the information, such as to present the information to only the appropriate audience or with the appropriate level of intrusiveness, and then sends the information to its conesponding output device for display. The Output Device Selector Module will also inform the User Characterization Module when output is to take place so that the model of the user condition can be updated accordingly.
Those skilled in the art will appreciate that the Format Modules may communicate with their corresponding output devices in a variety of ways, and that the body-mounted computer in the CDOS system may contain additional components or may lack some illustrated components. For example, there may not be a one-to-one mapping between Format Modules and output devices, functionality performed by the Output Device Selector Module and Format Modules may be incorporated together, and the creation of the model of the user condition may be performed by a different system than that which uses the information to present output information. There may also be multiple User Characterization or Output Device Selector Modules, such as one User Characterization Module for each relevant high-level condition variable. Alternately, external entities such as the application programs could add their own User Characterization, Output Device Selector or Format Modules, or could directly access the model of the user condition in order to perform presentation of output information. Accordingly, the present invention may be practiced with other computer system configurations.
Figure 3 is an illustrative example of a Model of User Condition 210. As is shown, the model reflects the condition of user X at time 14:22 hours on the displayed date. The illustrative model of the user condition includes a variety of user condition variables at different levels of abstraction, including low-level information directly from user sensors as well as higher-level abstract variables with characterized values that reflect a user's current physical and mental states. Historical and time- sensitive information can also be included, as shown by the variable illustrating the last user input performed by user X. Intermediate-level variables included in the model can be calculated from low-level input information such as sensor values. For example, the speed of the user could be calculated directly by a sensor such as a pedometer, or could be calculated indirectly via information over time from a GPS sensor. In addition, the Speed variable indicates that additional information can be included in the user model for each variable. In the case of the Speed variable, uncertainty about the exact value of the variable is demonstrated. Other calculated condition variables include an indication that the user is located in their office, is near their desk, and that there are no other people physically nearby. These factors can be determined in a variety of ways, such as via a motion sensor device located on the desk that is tracking the user and the absence of other individuals, or by the lack of any sounds from any other people via one or more microphones.
Higher-level condition variables can also be calculated, such as the user's current physical activities, the current user cognitive load, the desired level of privacy, and the desired scope of audience. Information from the microphone or directly from the cellular phone could indicate that the user is currently talking on their cellular phone, and the speed and motion sensor data could indicate that the user is walking. Since the user remains near his desk even though he is walking, the system can deduce that the user is pacing about his office or is walking on a treadmill (not shown). The User Activity variable demonstrates that variables can have multiple values, and that information such as a degree of belief or certainty in the value for a variable can be added and used by the system.
The Cognitive Load variable indicates a score of 77 out of 100, thus indicating a relatively high cognitive load due to the combination of the user walking and talking on the phone. Since it is unlikely that information presented by the system will be desired to be perceptible by the person on the other end of the phone, the desired Scope Of Audience variable indicates that only the user is currently appropriate to receive output information. Since the User Characterization Module was able to identify the other person on the phone as Doug Smith, an executive level colleague at user X's company (e.g., by voice recognition or the use of that person's name), the desired Level Of Pπvacy vanable indicates that if information is presented in a manner such that the other person can receive it (e g , through an external speaker), general information about the company as well as executive-level sensitive information can be presented Note that although low-level sensors such as a motion detector may have indicated that there are no other people physicallv nearby, when it was determined that the user was talking on a phone, additional information as added to the Nearby People vanable to indicate that someone is within audio perception of the user
The remaining displayed portions of the user condition model indicate that user preference information and externally supplied information can be included in the user condition model For example, the Application X-Factor 1 vanable has been supplied by application X, as well as a value for the vanable In this case, the value is a normal probability distnbution with a mean of 23 and a standard deviation of 3 In addition, previously supplied user preference information could indicate which output devices and which output formats are preferred by the user Alternately, the system could have automatically learned these preferences over time by observing user reactions to vanous outputs, as well as from explicit suggestions and ovemdes by the user Those skilled in the art will appreciate that the illustrated user condition model is merely illustrative and is not intended to limit the scope of the present invention The model may contain additional vaπables or may lack some illustrated vaπables, or may be represented without explicit condition vaπables at all
Figure 4 is an illustrative example of User Charactenzation Module 205 As is shown, the illustrated User Charactenzation Module is for user X and it includes a vanety of IF-THEN rules User condition vaπables are shown with angle brackets surrounding them, with some user condition v aπables (e g , Speakerphone Status) not shown in the illustrative model of user condition 210 in Figure 3 In addition to the IF- THEN rules, WHILE-THEN rules are also shown, as well as an application-specific rule (i e . the APPX rule) added by an external application The illustrative ser Characteπzation Module also indicates that the results portion of the rules (shown after the THEN statements) can set or modifv the alues of condition v aπables, such as bv absolute or percentage numeπcal amounts, and can indicate degrees of be et or uncertainty in values. Groups of people are shown in square brackets (e.g., Company Executives), and asterisks are wild-card characters that can match any information.
As mentioned previously, receiving input related to one user condition variable can cause multiple changes to propagate through the set of rules. For example, if input is received that indicates that the user condition variable
Desktop. Motion. Sensor.Human. Movement is true and the User Activity variable value indicates that user is seated, one of the rules shown indicates that the Nearby People variable will be modified (if necessary) to indicate that an "Unidentified Person" is physically nearby. Modifying the Nearby People variable can then affect the Level Of Privacy or Scope Of Audience user condition variables as shown by other rules. Those skilled in the art will appreciate that the illustrated User Characterization Module is merely illustrative and is not intended to limit the scope of the present invention. The model may contain additional rules, may lack some illustrated rules, or may be implemented without using rules at all. In addition, the test and/or results portions of rules can be implemented as invokable functions, including those provided by external entities such as application programs.
Figure 5 is an illustrative example of Output Device Selector Module 215. As is shown, the module is for user X and it maps each available output device to ratings for selected user condition variables and output information description factors. As is shown, some output devices which are available at times (e.g., pager 502. cellular telephone 504, and car radio 506) are not currently available. In addition, earpiece speaker 132 may not currently be able to receive output information if it is already in use (e.g., the user is listening to music). Alternately, new output information could preempt the current use of the earpiece speaker if necessary, or could instead share the use of the output device (e.g., outputting music to one ear and other information to the other ear if the earpiece speaker is part of headphones).
As is shown, the various output devices are rated with single values or a range of values for each factor. While textual values are used, those skilled in the art will appreciate that numeπcal or other types of rating systems could be used. In the illustrated embodiment, ranges may illustrate the device capabilities in different situations, with the ranges being restncted in any particular situation For example, the earpiece speaker can accommodate when the user has a very low cognitive load by adjusting the volume to be slightly louder than the ambient environment Alternately, even if the user has a high cognitive load, the earpiece speaker can interrupt the user if necessary for urgent information by using very loud volumes or distinctive tones In addition, the ratings can be adjusted to reflect the specific situation of this user For example, since the speaker 162 is located on the user's desk at work and other employees can frequently or always hear the speaker, the value for the desired Level Of Pnvacy may indicate that only business information be presented via the speaker Alternately, the system could present information DV sending it to the cellular telephone if the information is highly sensitive or it is important to interrupt the user However, if others are present around the user, frequent use of the cellular telephone can be highly intrusive to them (particularly in environments such as a lecture or a movie)
Those skilled in the art will appreciate that the illustrated Output Device Selector Module is merely illustrative and is not intended to limit the scope of the present invention The module may contain additional user condition vaπables and output information descnption factors, may lack some illustrated user condition vaπables and output information descnption factors, or may select output devices on which to present output information in an entirelv different manner In addition, some embodiments of the Output Device Selector Module may include specific logic, such as
IF-THEN rules, to be used in conjunction with the mapping of output devices as shown
Figure 6 is an exemplary flow diagram of an embodiment of the User
Characteπzation Routine 600 The User Charactenzation Routine retπeves stored information related to the user, including a default model of the user condition, receives vaπou-. types of input information related to the user or the user's environment, updates the model of the user condition to reflect the new information, and peπodically updates the model if no information has been received within a prescπbed time The routine begins at step 605 where stored information for the user is retπev ed. including a set of characteπzation rules to be used bv the routine The routine then creates a default model of the user condition, such as directlv from stored information or by applvmg the characteπzation rules to default user information that is available The routine then continues to step 610 to set a timer, and continues to step 615 to either receive input information or to receive a notification that the timer has expired
The routine continues to step 622 to determine if input information was received If so, the routine continues to step 625 to determine if the information received was information input to the computer by the user If so, the routine continues to step 630 to determine if the user input indicates that the user condition should be modified, such as bv setting a user preference or explicitly changing the value of a user condition vanable. If so, the routine continues to step 635 to satisfy the user request, and then returns to step 610 If it was instead determined in step 630 that the user input was not directed to the User Characteπzation Module, the routine continues to step 640 to forward the user input information to the appropnate destination (e g , an application program).
After step 640, or if it was determined in step 620 that the timer had expired or in step 625 that the information received was not user input, the routine continues to step 645 to execute the Characteπze User Subroutine to update the model of the current user condition After step 645, the routine continues to step 652 to store the updated user condition model, and then continues to step 655 to update the characteπzation rules if necessary The charactenzation rules can be updated in a vaπety of situations, such as if an external entity (e.g , an application) explicitly adds a new charactenzation rule or if the routine monitors the user's behavior and reactions in an attempt to learn more appropπate characteπzation rules. After step 655, the routine continues to step 660 to determine if there is more information to receive. If so. the routine returns to step 610, and if not, the routine ends at step 695 Figure 7 is an exemplary flow diagram of an embodiment of the
Characteπze User Subroutine 645 The subroutine is executed when information is received related to the user or the user's env ironment, or when a timer has expired indicating that no information has been receiv ed for a specified peπod of time When no information has been received for a peπod of "ime, the model of the user's current condition may need to o updated so that time-sensitive information can be updated in the model of the user condition The subroutine begins at step 705 where the current model of the user condition is retneved The suoroutine then continues to step 710 to retneve the current date and time In step 715, the subroutine determines whether information was received or if the timer has expired If the timer has expired, the subroutine continues to step 720 to examiner user condition vaπables tat represent time-sensitive information or histoπcal data and updates them if necessary The subroutine then continues to step 725 to determine if the current date and time tngger any charactenzation rules, and if so, the changes from these tnggered rules are propagated through the set of rules If it was instead determined in step 715 that information to be processed was receiv ed, the subroutine continues to step 730 to determine if a new user condition vanable has been defined, such as bv an application program, and if so continues to step 750 If a new user condition vanable has not been defined, however, the subroutine continues to step 735 to determine if a new user charactenzation rule is being added. such as by an application program If so, the subroutine continues to step 740 to add the new charactenzation rule, determine if current v alues for any user condition vanables tπgger the rule, and if so propagates any changes from the tnggered rules through the set of rules If it was instead determined in step 735 that a new charactenzation rule is not being defined, the subroutine continues to step 745 to determine if the current input information or the current date and time tngger any rules, and if so, changes from those tnggered rules are propagated throughout the set of rules In addition to information received directly from the user, sensors, or application programs, this input information can also be a notification from the Output Device Selector that indicates output information is currently being presented to the user After steps 725, 730, 740, or 745 the subroutine continues to step 750 to store any changes in user condition vanables and their values, as well as the new date and time, in an updated model of the user condition The subroutine then continues to step 795 and returns Those skilled in the art w ill appreciate that a v ariety of types of information related to the user and the user's env ironment can be received, and that the User Characteπzation Routine and the Characteπze User Subroutine can process this information in a vaπety of ways, including other man with sets of IF-THEN rules
Figure 8 is an exemplary flow diagram of an embodiment of the Output
Device Selector Routine 800 The Output Dev ice Selector Routine receives output information to be presented to the user, retπev es cunent charactenzed information about the user from the model of the user condition, determines whether the output information should be presented to the user at this time and if so on which output device and in what format, and then notifies the User Characteπzation Module when output information is presented to the user The routine begins at step 805 where output information to be presented to the user is recei ed or an indication that a timer has expired occurs The routine continues to step 810 to determine if a timer has expired
When output information cannot be currently presented to the user (e g , no satisfactory output device is available or presentation to the user could be dangerous or inappropπate), the presentation is deferred and a timer is set to indicate when to rev lew presenting the information Thus, if it is determined in step 810 that a timer has expired, the routine continues to step 815 to retπeve the deferred output information for the timer, as well as any descnption information for the defened output information If it is instead determined in step 810 that new output information to be presented has been receιved, the routine continues to step 820 where descnption information for the output information is optionally received
After steps 815 or 820, the routine continues to step 825 to retπeve relevant information from the cuπent model of the user condition The routine then continues to step 830 to determine whether to cunently present the output information to the user. In the illustrated embodiment, this determination is made using the user condition vanables of cognitive load, desired level of pπvacy, and desired scope of audience. In addition, available descnption information which indicates the importance and the deferability of the output information and the consequences of the user ignoπng or not receiving the output information are considered, as is any user preference information Cunent alues for these user condition vanables and descnption factors as well as whether av ailable output dev ices can support the necessary formatting oi information (e g , presenting information to the appropπate scope of audience or at the appropπate level of intrusiveness for the user's cognitive load), are thus used in the determination. Those skilled in the art will appreciate that other factors can be used for this determination or that the determination could be made in other ways. The routine then continues to step 835 to determine whether the presentation of the information is to be deferred or not. If the presentation is to be deferred, the routine continues to step 840 to store the output information as well as its description information, and to set a timer for the information at which time the presentation of the output information will be reconsidered. If it is instead determined in step 835 that the information presentation is not to be deferred, the routine continues to step 845 where an available output device is selected. In the illustrated embodiment, the output device whose information display capabilities and ratings best match the user condition variables and description information factors of interest is chosen. The routine then continues to step 850 to execute the Format .And Present Output Information Subroutine, and then continues to step 855 to notify the User Characterization Module of the presentation of the output information. After step 840 or step 855, the routine continues to step 860 to determine if there are currently timers set or there is more output information to be received. If so, the routine returns to step 805, and if not the routine ends at step 895. Figure 9 is an exemplary flow diagram of an embodiment of the Format
And Present Output Information Subroutine 850. The subroutine receives output information to be presented and its description information, receives relevant user condition variables and user preference information, selects a user sense to which the output information will be presented (if the output device supports more than one). selects appropπate formatting with which to present the output information, and presents the output information to the user. The subroutine begins at step 905 where output information is received, as well as the descnption information factors, user condition variables, and relevant user preference information. The subroutine continues at step 910 to select a user sense that is supported by the selected output device. In step 915, the subroutine selects formatting for the output information that is appropπate for the user condition vaπables. o1 ut information descnption, and user preferences Those skilled in the art will appreciate the formatting of the output information will vary with each user sense (e g , adjusting volume for the audio sense and adjusting pressure for the tactile sense), as well as with the specific output device. After the formatting for the output information is selected, the subroutine continues to step 920 to present the output information to the user with the selected formatting. If the Scope Of Audience and Level of Pnvacy user condition vanables indicate that the information can be presented to other people cunently present and the selected output device supports such presentation, the information will also be presented to these other people. After step 920, the subroutine continues to step 995 and returns.
Those skilled in the art will appreciate that the selection of an output device and the formatting of the output information for that device can be performed in a vaπety of ways. For example, other user condition vaπables and descnption information factors could be used, or the selection could be made without resort to such information. For example, in one embodiment, the user could explicitly indicate the output device and formatting desired for some or all pieces of output information (e g , in response to an notification from the system), or another er" ty (e g , an application program supplying the output information) could explicitly designate the output device and/or formatting.
From the foregoing it will be appreciated that, although specific embodiments of the invention have been descπbed herein for purposes of illustration, various modifications may be made without deviating from the spmt and scope of the invention. Accordingly, the invention is not limited except as by the appended claims

Claims

1 A computer-implemented method for presenting output information to a user, the method comprising receiving information about a modeled property of the user which affects appropriateness of presenting output information to the user, and presenting the output information to the user by, selecting an output device capable of presenting the output information in accordance with the modeled user property, and presenting the output information on the selected output device in accordance with the modeled user property
2 The method of claim 1 wherein a plurality of output devices are available for presenting information to the user, wherein the modeled property of the user is one of a level of pπvacy desired by the user indicating a group of people allowed to perceive information presented by the computer a scope of audience desired by the user indicating how many people are intended to perceive information presented by the computer, or a cognitive load of the user indicating ability of the user to devote attention to the computer, and including (a) monitoring the user to collect information about a current state of the user, (b) modeling a current user condition based on the collected information by determining the current level of privacv desired by the user, the current scope of audience desired by the user, or the current cognitive load of the user, and (c) receiving the output information to be presented to the user, and wherein the selecting of the output device includes selecting one of the plurality of available output devices such that information presentation capabilities of the selected output device support the determined current desired level of privacy, the determined current desired scope of audience, and the determined current cognitive load
3 The method of claim 2 wherein the modeling of the current user condition is additionally based on collected information related to a surrounding environment
4 The method of claim 2 wherein the monitoring of the user is performed by sensors worn by the user
5 The method of claim 2 wherein the monitoring of the user is performed by sensors remote from the user
6 The method of claim 2 wherein the collected information includes information about a current physiological state of the user
7 The method of claim 2 wherein the collected information includes information about current activities of the user
8 The method of claim 2 wherein the computer is a wearable computer
9 The method of claim 2 wherein an operating system of the computer performs the method
10 The method of claim 2 wherein the output information is received from a distinct program executing on the computer
1 1 The method of claim 2 wherein the output information is received from another computer
12 The method of claim 2 wherein the output information is generated bv the computer during the monitoring or the modeling
13 The method of claim 2 including receiving description information for the output information describing importance and deferability of the output information, and wherein the presenting of the output information to the user is additionally performed in a manner consistent with the description information
14 The method of claim 2 wherein the presenting of the output information to the user is deferred when the output information cannot be presented on any of the available output devices in a manner consistent with the modeled current user condition
15 The method of claim 2 wherein the available output devices include two display devices with different information presentation capabilities regarding level of privacy, scope of audience, and cognitive load, and wherein the selected output device is the display device whose information presentation capabilities best match the modeled current user condition
16 The method of claim 2 wherein information can be presented with the available output devices to the user via at least two user senses, and wherein the selecting includes determining the user sense via which the output information will be presented
17 The method of claim 1 wherein the received information includes information related to a current condition of the user, and including modeling the user property based on the received information
18 The method of claim 17 including monitoring the user to collect the received information
19 The method of claim 1 wherein the received information includes information related to a surrounding environment, and including modeling the user property based on the received information
20 The method of claim 19 including monitoring the surrounding environment to collect the received information
21 The method of claim 1 wherein the modeled user property is based on algorithmic modeling
22 The method of claim 1 including before the presenting of the output information, receiving the output information
23 The method of claim 22 wherein an information provider transmits information to various computers within a transmission range of the information provider, wherein the method is performed by a transportable computer transported by the user, wherein the user transports the transportable computer within the transmission range, and wherein the received output information is the transmitted information
24 The method of claim 1 including receiving description information indicating importance or deferability of the output information, and wherein the selecting is additionally performed in accordance with the description information
25 The method of claim 1 including deferring the presenting of the output information when no selectable output device is capable of presenting the output information in accordance with the modeled user property
26 The method of claim 1 wherein the modeled user property is an indication of a degree of interruptibilitv of current activities of the user
27 The method of claim 26 wherein the degree of interruptibility of the current activities is influenced by information being received by the user via one user sense in conjunction with the current activities, and wherein the presenting of the output information is selected to be via a distinct user sense
28 The method of claim 26 including deferring the presenting of the output information when the degree of interruptibility of the current activities is low
29 The method of claim 26 wherein the degree of interruptibility of the current activities is low when the current activities involve a risk of harm to the user.
30 The method of claim 1 wherein the modeled user property is an indication of ability of the user to devote attention to the presenting of the output information
31 The method of claim 30 wherein the ability of the user to devote attention is influenced by information being received by the user via one user sense, and wherein the presenting of the output information is selected to be via a distinct user sense
32 The method of claim 30 including deferring the presenting of the output information when the ability of the user to devote attention is low
33 The method of claim 30 wherein the indication of the ability of the user to devote attention is an estimate of an amount of attention devoted by the usei to other current activities
34 The method of claim 1 wherein the modeled user property is a preference of the user for an amount of people to perceive presented information
35 The method of claim 34 wherein the selected output device includes a range of information presentation capabilities, wherein others can perceive information presented using some of the information presentation capabilities, and including selecting those information presentation capabilities to be used for the presenting when the modeled user property indicates that the others are within the amount of people to perceive the output information
36 The method of claim 34 wherein the selected output device includes a range of information presentation capabilities, wherein others nearby cannot perceive information presented using some of the information presentation capabilities, and including selecting those information presentation capabilities to be used for the presenting when the modeled user property indicates that the others exceed the amount of people to perceive the output information
37 The method of claim 1 wherein the modeled user property is a preference of the user for how to receive sensitive information
38 The method of claim 37 wherein the selected output device includes a range of information presentation capabilities, wherein others can perceive information presented using some of the information presentation capabilities, and including selecting those information presentation capabilities to be used for the presenting when the modeled user property indicates that sensitivity of the output information allows the others to perceive the output information
39 The method of claim 37 wherein the selected output device includes a range of information presentation capabilities, wherein others nearby cannot perceive information presented using some of the information presentation capabilities, and including selecting those information presentation capabilities to be used for the presenting when the output information is sufFicientlv sensitive that the modeled user property indicates that the others are not allowed to perceive the output information
40 The method of claim 1 wherein the modeled user property is a preference of the user for receiving information during current activities of the user
41 The method of claim 40 including deferring the presenting of the output information when the preference of the user is to not receive the output information during the current activities
42 The method of claim 40 including presenting the output information non-intrusively when the preference of the user is to not receive output information during the current activities
43 The method of claim 1 wherein the modeled user property is an indication of appropriateness of presenting the output information in a manner perceivable by others
44 The method of claim 43 wherein the selected output device includes a range of information presentation capabilities, wherein others cannot perceive information presented using some of the information presentation capabilities, and including selecting those information presentation capabilities to be used for the presenting when the modeled user property indicates that presenting the output information in a manner perceivable by the others is not appropriate.
45. The method of claim 43 including deferring the presenting of the output information when the modeled user property indicates that presenting the output information in a manner perceivable by others is not appropriate and available output devices cannot present the output information in a manner not perceivable by the others.
46. The method of claim 1 wherein the selected output device includes a range of information presentation capabilities, and including formatting the output information before the presenting, the formatting to select information presentation capabilities of the selected output device to be used such that the selected inrormation presentation capabilities are ~ - nsistent with the modeled user property.
47. The method of claim 1 wherein a plurality of output devices are available to present information via distinct senses of the user, and wherein the selecting of the output device includes determining a user sense which is capable of presenting the output information in accordance with the modeled user property.
18. The method of claim 1 wherein the selected output device has only a single physical mechanism for presenting the output information to the user.
49. The method of claim 1 wherein the computer performing the method is transportable by the user, wherein fixed output devices become available to the computer when the user transports the computer near the fixed output devices, and wherein the selected output device is a fixed output device.
50. The method of claim 1 wherein a user computer performing the method is transportable by the user, and wherein the user computer can communicate with other devices within a transmission range of the user computer.
51. The method of claim 50 wherein the user computer communicates with another computer, and wherein the selected output device is an output device of the another computer.
52. The method of claim 50 wherein the received information is from one of the other devices.
53. The method of claim 50 wherein the output information to be presented is received from one of the other devices.
54. The method of claim 1 including after the presenting of the output information, revising the modeled user property based on the presenting.
55. The method of claim 1 wherein the modeled user property indicates capabilities of the user for receiving presented output information.
56. The method of claim 55 wherein the capabilities indicate that a physical disability of the user prevents the user from perceiving some types of presentations of information, and wherein the presenting of the output information is in a manner perceivable by the user.
57. The method of claim 1 including modeling properties of the user which affect appropriateness of presenting output information to the user, the modeling including: receiving information about a current state of the user; and for at least one of a plurality of properties of the user, determining whether the received information relates to the property; and when the received information relates to the property, determining a current value for the property based on the received information
58 The method of claim 57 including monitoring the user to obtain the received information
59 The method of claim 57 wherein multiple pieces of information are received which relate to a property of the user, wherein the multiple pieces of information are inconsistent as to the current value for the property, and wherein the determining of the current value involves mediating the inconsistencies
60 The method of claim 57 wherein a rating indicating quality of the received information is received, and wherein the quality rating is reflected in the determined current values for the properties which relate to the received information
61 The method of claim 1 wherein the computer is able to output information to a first display device and a second display device, the first and second display devices having different display characteristics, wherein the selecting of the output device includes selecting either the first display device or the second display device based on a predicted preference of the user, the predicted preference indicating on which of the display devices the user would prefer to receive the output information, and wherein the presenting of the output information on the selected output device in accordance with the modeled user property includes presenting the output information on the selected display device consistently with the predicted preference
62 The method of claim 61 including predicting a current value of the preference before the selecting
63 The method of claim 62 including monitoring the user to obtain information for the predicting
64 The method of claim 61 wherein the predicted preference is a predicted mental state of the user, and wherein a mapping between the predicted mental state and the display devices is used for the selecting, the mapping indicating which of the display devices are suitable for presenting information in accordance with various predicted mental states
65 The method of claim 64 including after the presenting, monitoring reaction of the user to the presenting, and revising the mapping based on the monitored reaction to enhance user reaction to future presentations of information
66 The method of claim 1 wherein the computer is a wearable computer worn by the user and has access to a plurality of output devices from which the user can receive information, wherein the modeled property of the user includes at least one of an ability of the user to currently receive the output information and a desire of the user of how to currently receive the output information, and including monitoring the user to collect information, and characterizing the ability of the user to currently receive the output information and the desire of the user of how to currently receive the output information based on the collected information, and wherein the selecting of the output device includes selecting one of the plurality of output devices such that information presentation capabilities of the selected output device support the characterized ability and desire
67 The method of claim 66 including receiving description information indicating importance or deferability of the output information, and wherein the selecting is additionally performed in accordance with the description information
68 The method of claim 66 including receiving the output information from an application program executing on the wearable computer
69 A computer-readable medium containing instructions for presenting output information to a user by receiving information about a modeled property of the user which affects appropriateness of presenting output information to the user, selecting an output device capable of presenting the output information in accordance with the modeled user property, and presenting the output information on the selected output device in accordance with the modeled user property
70 The computer-readable medium of claim 69 wherein the at least one modeled characteristic is based on collected information related to the user
71 The computer-readable medium of claim 70 wherein the computer system is further controlled by monitoring the user to obtain the collected information
72 The computer-readable medium of claim 69 wherein the computer system is further controlled by receiving the output information; and receiving description information indicating importance or deferability of the output information, and wherein the selecting is additionally performed in accordance with the description information.
73 The computer-readable medium of claim 69 wherein the selected output device includes a range of information presentation capabilities, and wherein the computer system is further controlled by formatting the output information before the presenting, the formatting to select information presentation capabilities of the selected output device to be used such that the selected information presentation capabilities are consistent with the at least one modeled characteristic
74 A computer system for presenting output to a user, comprising. an output device selector module that receives information about a modeled property of the user which affects appropriateness of presenting output to the user, that selects an output device capable of presenting the output in accordance with the modeled property, and that presents the output on the selected output device in accordance with the modeled property
75. The computer system of claim 74 further comprising the selected output device.
76 The computer system of claim 74 further comprising a model of a current condition of the user, the model including a plurality of user properties including the modeled property.
77 The computer system of claim 74 wherein the computer system further comprises a user characterization module that generates the modeled property based on collected information related to the user or to a surrounding environment.
78 The computer system of claim 77 wherein the user characterization module further monitors the user or the surrounding environment to obtain the collected information
79 The computer system of claim 74 wherein the output device selector module further receives description information indicating importance or deferability of the output, and wherein the selecting of the output device is additionally performed in accordance with the description information
80 The computer system of claim 74 wherein the selected output device includes a range of information presentation capabilities, and further including a format module that formats the output before the presenting, the formatting to select information presentation capabilities of the selected output device to be used such that the selected information presentation capabilities are consistent with the modeled property
81 The computer system of claim 74 wherein the received information is current physiological information about the user indicating a health condition, and wherein the presenting of output is to another person to alert the another person of a health problem detected from the current physiological information
82 The computer system of claim 74 wherein the received information is current physiological information about the user indicating a health condition, and wherein the presenting of output is to administer medical care for the health condition
83 A data structure stored in the memory of a computer for use in presenting output information to a user, the data structure containing modeled information regarding the user, the data structure used by extracting information from the data structure including a current value of a modeled ability of the user to currently receive the output information or of a modeled desire of the user of how to currently receive the output information, selecting an output device capable of presenting the output information in accordance with the extracted information, and presenting the output information on the selected output device in accordance with the extracted information
84 The data structure of claim 83 including a defined interface to receive updates to the modeled information from another computer
85 The data structure of claim 83 wherein the data structure further contains a plurality of modeling rules useful for determining based on received information about the user a current value for the modeled ability or desire
86 The data structure of claim 85 wherein the modeling rules are useful for mediating inconsistencies between multiple pieces of information which relate to the modeled ability or desire so as to determine the current value
87 A computer program adapted to model properties of a user for use when presenting output information to the user when the program is executed on a computer, the execution to perform a method comprising receiving information about a current state of the user, and for at least one of a plurality of properties of the user which affects appropriateness of presenting output information to the user, modeling the property by determining whether the received information relates to the property, and when the received information relates to the propert , determining a current value for the property based on the received information, so that the determined current values of the modeled user properties can be used for presenting output information to the user in an appropriate manner
88 The computer program of claim 87 wherein the method includes receiving output information to be presented in accordance with the determined current values of the modeled user properties, and presenting the output information to the user in accordance with the modeled user properties
89. The computer program of claim 87 wherein the method includes when another program has output information to present to the user, supplying the determined current values of the modeled user properties to the another program so that the output information can be presented to the user by the another program in accordance with the modeled user properties.
90. The computer program of claim 87 wherein the determining of the current value for the property is based on a plurality of modeling rules.
91. The computer program of claim 90 wherein when the user is in a specified class of users, using modeling rules specialized for the specified class.
92. The computer program of claim 90 wherein when the user is not in a pre-determined class of users, using default modeling rules.
93. The computer program of claim 90 wherein the method includes updating the modeling rules based on the received information so as to better model the user.
94. The computer program of claim 87 wherein the computer can communicate with another computer, the another computer able to obtain information about the user via input devices of the another computer, and wherein the received information is obtained information about the user from the another computer.
95. The computer program of claim 87 wherein the computer can communicate with another computer, and including receiving from the another computer information about properties of the user to be added to the modeled user properties, the added properties distinct from the modeled user properties. 96 The computer program of claim 87 wherein the computer can communicate with another computer, and including receiving from the another computer a value for one of the modeled user properties to be stored as the determined current value for the one property
97 The computer program of claim 87 wherein the method includes monitoring the user to obtain the received information
98 The computer program of claim 87 wherein multiple pieces of information are received which relate to a property of the user, wherein the multiple pieces of information are inconsistent as to the current value for the property, and wherein the determining of the current value involves mediating the inconsistencies
99 The computer program of claim 87 wherein a rating indicating quality of the received information is received, and wherein the quality rating is reflected in the determined current values for the properties which relate to the received information
PCT/US1999/029952 1998-12-18 1999-12-15 Method and system for controlling presentation of information to a user based on the user's condition WO2000036493A1 (en)

Priority Applications (3)

Application Number Priority Date Filing Date Title
AU21902/00A AU2190200A (en) 1998-12-18 1999-12-15 Method and system for controlling presentation of information to a user based onthe user's condition
DE69902487T DE69902487T2 (en) 1998-12-18 1999-12-15 DEVICE AND METHOD FOR CONTROLLING A INFORMATION PRESENTATION DEPENDING ON USER STATES
EP99966337A EP1151372B1 (en) 1998-12-18 1999-12-15 Method and system for controlling presentation of information to a user based on the user's condition

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
US09/216,193 1998-12-18
US09/216,193 US6466232B1 (en) 1998-12-18 1998-12-18 Method and system for controlling presentation of information to a user based on the user's condition

Publications (1)

Publication Number Publication Date
WO2000036493A1 true WO2000036493A1 (en) 2000-06-22

Family

ID=22806105

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/US1999/029952 WO2000036493A1 (en) 1998-12-18 1999-12-15 Method and system for controlling presentation of information to a user based on the user's condition

Country Status (5)

Country Link
US (3) US6466232B1 (en)
EP (2) EP1276037A3 (en)
AU (1) AU2190200A (en)
DE (1) DE69902487T2 (en)
WO (1) WO2000036493A1 (en)

Cited By (17)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2003100585A2 (en) * 2002-05-24 2003-12-04 Koninklijke Philips Electronics N.V. Context-aware portable device
EP1388093A1 (en) * 2001-05-15 2004-02-11 Nokia Corporation Method and business process to maintain privacy in distributed recommendation systems
EP1452988A1 (en) * 2003-02-28 2004-09-01 Microsoft Corporation Bounded-deferral policies for guiding the timing of alerting, interaction and communications using local sensory information
EP1182541A3 (en) * 2000-08-22 2005-11-30 Siemens Aktiengesellschaft System and method for combined use of different display/apparatus types with system controlled context dependant information representation
EP1606709A1 (en) * 2003-03-26 2005-12-21 Microsoft Corporation Extensible user context system for delivery of notifications
US7243130B2 (en) 2000-03-16 2007-07-10 Microsoft Corporation Notification platform architecture
US8413167B2 (en) * 2007-10-01 2013-04-02 Microsoft Corporation Notifying a user of access to information by an application
WO2015099912A1 (en) * 2013-12-23 2015-07-02 Google Inc. Displaying private information on personal devices
US9183306B2 (en) 1998-12-18 2015-11-10 Microsoft Technology Licensing, Llc Automated selection of appropriate information based on a computer user's context
WO2016028538A1 (en) * 2014-08-20 2016-02-25 Google Inc. Displaying private information on personal devices
US9372555B2 (en) 1998-12-18 2016-06-21 Microsoft Technology Licensing, Llc Managing interactions between computer users' context models
US9443037B2 (en) 1999-12-15 2016-09-13 Microsoft Technology Licensing, Llc Storing and recalling information to augment human memories
US9497309B2 (en) 2011-02-21 2016-11-15 Google Technology Holdings LLC Wireless devices and methods of operating wireless devices based on the presence of another person
US9559917B2 (en) 1998-12-18 2017-01-31 Microsoft Technology Licensing, Llc Supplying notifications related to supply and consumption of user context data
US9832187B2 (en) 2014-01-07 2017-11-28 Google Llc Managing display of private information
US10277479B2 (en) 2010-05-11 2019-04-30 Nokia Technologies Oy Method and apparatus for determining user context
US10311249B2 (en) 2017-03-31 2019-06-04 Google Llc Selectively obscuring private information based on contextual information

Families Citing this family (528)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US8152198B2 (en) * 1992-05-05 2012-04-10 Automotive Technologies International, Inc. Vehicular occupant sensing techniques
US7570785B2 (en) * 1995-06-07 2009-08-04 Automotive Technologies International, Inc. Face monitoring system and method for vehicular occupants
EP0816986B1 (en) * 1996-07-03 2006-09-06 Hitachi, Ltd. System for recognizing motions
JP2000194726A (en) * 1998-10-19 2000-07-14 Sony Corp Device, method and system for processing information and providing medium
US7881936B2 (en) 1998-12-04 2011-02-01 Tegic Communications, Inc. Multimodal disambiguation of speech recognition
US8938688B2 (en) * 1998-12-04 2015-01-20 Nuance Communications, Inc. Contextual prediction of user words and user actions
US7712053B2 (en) 1998-12-04 2010-05-04 Tegic Communications, Inc. Explicit character filtering of ambiguous text entry
US7679534B2 (en) * 1998-12-04 2010-03-16 Tegic Communications, Inc. Contextual prediction of user words and user actions
US8225214B2 (en) 1998-12-18 2012-07-17 Microsoft Corporation Supplying enhanced computer user's context data
US7225229B1 (en) 1998-12-18 2007-05-29 Tangis Corporation Automated pushing of computer user's context data to clients
US6842877B2 (en) 1998-12-18 2005-01-11 Tangis Corporation Contextual responses based on automated learning techniques
US6920616B1 (en) 1998-12-18 2005-07-19 Tangis Corporation Interface for exchanging context data
US7779015B2 (en) 1998-12-18 2010-08-17 Microsoft Corporation Logging and analyzing context attributes
US6483484B1 (en) * 1998-12-18 2002-11-19 Semiconductor Energy Laboratory Co., Ltd. Goggle type display system
US7231439B1 (en) 2000-04-02 2007-06-12 Tangis Corporation Dynamically swapping modules for determining a computer user's context
US8181113B2 (en) 1998-12-18 2012-05-15 Microsoft Corporation Mediating conflicts in computer users context data
US7046263B1 (en) 1998-12-18 2006-05-16 Tangis Corporation Requesting computer user's context data
US7073129B1 (en) 1998-12-18 2006-07-04 Tangis Corporation Automated selection of appropriate information based on a computer user's context
US8636648B2 (en) 1999-03-01 2014-01-28 West View Research, Llc Endoscopic smart probe
US10973397B2 (en) 1999-03-01 2021-04-13 West View Research, Llc Computerized information collection and processing apparatus
US7596606B2 (en) * 1999-03-11 2009-09-29 Codignotto John D Message publishing system for publishing messages from identified, authorized senders
US6999955B1 (en) * 1999-04-20 2006-02-14 Microsoft Corporation Systems and methods for estimating and integrating measures of human cognitive load into the behavior of computational applications and services
US7789742B1 (en) * 1999-05-12 2010-09-07 Wilbert Q. Murdock Smart golf club multiplayer system for the internet
US6434527B1 (en) * 1999-05-17 2002-08-13 Microsoft Corporation Signalling and controlling the status of an automatic speech recognition system for use in handsfree conversational dialogue
US6931384B1 (en) 1999-06-04 2005-08-16 Microsoft Corporation System and method providing utility-based decision making about clarification dialog given communicative uncertainty
US7103806B1 (en) 1999-06-04 2006-09-05 Microsoft Corporation System for performing context-sensitive decisions about ideal communication modalities considering information about channel reliability
US7389351B2 (en) * 2001-03-15 2008-06-17 Microsoft Corporation System and method for identifying and establishing preferred modalities or channels for communications based on participants' preferences and contexts
US8065155B1 (en) 1999-06-10 2011-11-22 Gazdzinski Robert F Adaptive advertising apparatus and methods
US7711565B1 (en) * 1999-06-10 2010-05-04 Gazdzinski Robert F “Smart” elevator system and method
US7194681B1 (en) * 1999-07-30 2007-03-20 Microsoft Corporation Method for automatically assigning priorities to documents and messages
US6714967B1 (en) 1999-07-30 2004-03-30 Microsoft Corporation Integration of a computer-based message priority system with mobile electronic devices
US6622160B1 (en) 1999-07-30 2003-09-16 Microsoft Corporation Methods for routing items for communications based on a measure of criticality
US6527711B1 (en) 1999-10-18 2003-03-04 Bodymedia, Inc. Wearable human physiological data sensors and reporting system therefor
US8578439B1 (en) * 2000-01-28 2013-11-05 Koninklijke Philips N.V. Method and apparatus for presentation of intelligent, adaptive alarms, icons and other information
US7249159B1 (en) * 2000-03-16 2007-07-24 Microsoft Corporation Notification platform architecture
US8701027B2 (en) 2000-03-16 2014-04-15 Microsoft Corporation Scope user interface for displaying the priorities and properties of multiple informational items
US7634528B2 (en) 2000-03-16 2009-12-15 Microsoft Corporation Harnessing information about the timing of a user's client-server interactions to enhance messaging and collaboration services
US7565403B2 (en) * 2000-03-16 2009-07-21 Microsoft Corporation Use of a bulk-email filter within a system for classifying messages for urgency or importance
US8024415B2 (en) 2001-03-16 2011-09-20 Microsoft Corporation Priorities generation and management
US7743340B2 (en) * 2000-03-16 2010-06-22 Microsoft Corporation Positioning and rendering notification heralds based on user's focus of attention and activity
US6847924B1 (en) * 2000-06-19 2005-01-25 Ncr Corporation Method and system for aggregating data distribution models
US20020004817A1 (en) * 2000-03-29 2002-01-10 Thai-Lai Pham Method and apparatus for augmenting a device with surrounding resources for delivering services
WO2001075676A2 (en) * 2000-04-02 2001-10-11 Tangis Corporation Soliciting information based on a computer user's context
US7464153B1 (en) 2000-04-02 2008-12-09 Microsoft Corporation Generating and supplying user context data
US6690351B1 (en) * 2000-04-06 2004-02-10 Xybernaut Corporation Computer display optimizer
US6938024B1 (en) * 2000-05-04 2005-08-30 Microsoft Corporation Transmitting information given constrained resources
FR2809844B1 (en) * 2000-05-31 2002-11-22 Wokup Sa MULTI-TERMINAL PUBLICATION SYSTEM AND IMPLEMENTING METHOD THEREOF
BRPI0414359A (en) * 2000-06-16 2006-11-14 Bodymedia Inc body weight monitoring and management system and other psychological conditions that include interactive and personalized planning, intervention and reporting
US20130158367A1 (en) * 2000-06-16 2013-06-20 Bodymedia, Inc. System for monitoring and managing body weight and other physiological conditions including iterative and personalized planning, intervention and reporting capability
US7261690B2 (en) * 2000-06-16 2007-08-28 Bodymedia, Inc. Apparatus for monitoring health, wellness and fitness
US7689437B1 (en) 2000-06-16 2010-03-30 Bodymedia, Inc. System for monitoring health, wellness and fitness
US20060122474A1 (en) * 2000-06-16 2006-06-08 Bodymedia, Inc. Apparatus for monitoring health, wellness and fitness
US8086672B2 (en) * 2000-06-17 2011-12-27 Microsoft Corporation When-free messaging
JP4547777B2 (en) * 2000-06-19 2010-09-22 コニカミノルタビジネステクノロジーズ株式会社 E-mail management apparatus, e-mail processing system, e-mail processing method, and computer-readable recording medium recording e-mail processing program
ES2260245T3 (en) * 2000-06-23 2006-11-01 Bodymedia, Inc. SYSTEM TO CONTROL HEALTH, WELFARE AND EXERCISE.
WO2002029603A1 (en) * 2000-09-30 2002-04-11 Intel Corporation Method, apparatus, and system for determining information representations and modalities based on user preferences and resource consumption
US20020045994A1 (en) * 2000-10-13 2002-04-18 Thomas G. Hampton Remote signal analyzer, classifier, and database generator
US20020151297A1 (en) * 2000-10-14 2002-10-17 Donald Remboski Context aware wireless communication device and method
GB2386724A (en) * 2000-10-16 2003-09-24 Tangis Corp Dynamically determining appropriate computer interfaces
US20020054130A1 (en) 2000-10-16 2002-05-09 Abbott Kenneth H. Dynamically displaying current status of tasks
US20020093461A1 (en) * 2000-11-08 2002-07-18 Xtronx, Inc. Outdoor display and information system
US7844666B2 (en) 2000-12-12 2010-11-30 Microsoft Corporation Controls and displays for acquiring preferences, inspecting behavior, and guiding the learning and decision policies of an adaptive communications prioritization and routing system
US20020075306A1 (en) * 2000-12-18 2002-06-20 Christopher Thompson Method and system for initiating communications with dispersed team members from within a virtual team environment using personal identifiers
JP2002282543A (en) * 2000-12-28 2002-10-02 Sony Computer Entertainment Inc Object voice processing program, computer-readable recording medium with object voice processing program recorded thereon, program execution device, and object voice processing method
US6745193B1 (en) 2001-01-25 2004-06-01 Microsoft Corporation System and method for defining, refining, and personalizing communications policies in a notification platform
US6901398B1 (en) 2001-02-12 2005-05-31 Microsoft Corporation System and method for constructing and personalizing a universal information classifier
US20040201695A1 (en) * 2001-02-15 2004-10-14 Rei Inasaka System for delivering news
US8452259B2 (en) 2001-02-20 2013-05-28 Adidas Ag Modular personal network systems and methods
AU2002255568B8 (en) 2001-02-20 2014-01-09 Adidas Ag Modular personal network systems and methods
SE518484C2 (en) * 2001-02-27 2002-10-15 Peder Holmbom Apparatus and method for disinfecting water for medical or dental purposes
US7302634B2 (en) * 2001-03-14 2007-11-27 Microsoft Corporation Schema-based services for identity-based data access
US7024662B2 (en) 2001-03-14 2006-04-04 Microsoft Corporation Executing dynamically assigned functions while providing services
US7251696B1 (en) 2001-03-15 2007-07-31 Microsoft Corporation System and methods enabling a mix of human and automated initiatives in the control of communication policies
US6988132B2 (en) * 2001-03-15 2006-01-17 Microsoft Corporation System and method for identifying and establishing preferred modalities or channels for communications based on participants' preferences and contexts
US7330895B1 (en) * 2001-03-15 2008-02-12 Microsoft Corporation Representation, decision models, and user interface for encoding managing preferences, and performing automated decision making about the timing and modalities of interpersonal communications
GB0107689D0 (en) * 2001-03-28 2001-05-16 Ncr Int Inc Self service terminal
US7512940B2 (en) * 2001-03-29 2009-03-31 Microsoft Corporation Methods and apparatus for downloading and/or distributing information and/or software resources based on expected utility
US7757250B1 (en) 2001-04-04 2010-07-13 Microsoft Corporation Time-centric training, inference and user interface for personalized media program guides
US6947935B1 (en) * 2001-04-04 2005-09-20 Microsoft Corporation Training, inference and user interface for guiding the caching of media content on local stores
US7039642B1 (en) * 2001-05-04 2006-05-02 Microsoft Corporation Decision-theoretic methods for identifying relevant substructures of a hierarchical file structure to enhance the efficiency of document access, browsing, and storage
US7107254B1 (en) 2001-05-07 2006-09-12 Microsoft Corporation Probablistic models and methods for combining multiple content classifiers
AU2002346211B2 (en) 2001-06-27 2008-06-12 Sony Corporation Integrated circuit device, information processing device, information recording device memory management method, mobile terminal device, semiconductor integrated circuit device, and communication method using mobile terminal device
US7043506B1 (en) 2001-06-28 2006-05-09 Microsoft Corporation Utility-based archiving
US7493369B2 (en) * 2001-06-28 2009-02-17 Microsoft Corporation Composable presence and availability services
US7409423B2 (en) * 2001-06-28 2008-08-05 Horvitz Eric J Methods for and applications of learning and inferring the periods of time until people are available or unavailable for different forms of communication, collaboration, and information access
US7689521B2 (en) * 2001-06-28 2010-03-30 Microsoft Corporation Continuous time bayesian network models for predicting users' presence, activities, and component usage
US7089226B1 (en) 2001-06-28 2006-08-08 Microsoft Corporation System, representation, and method providing multilevel information retrieval with clarification dialog
US7233933B2 (en) 2001-06-28 2007-06-19 Microsoft Corporation Methods and architecture for cross-device activity monitoring, reasoning, and visualization for providing status and forecasts of a users' presence and availability
US7519529B1 (en) 2001-06-29 2009-04-14 Microsoft Corporation System and methods for inferring informational goals and preferred level of detail of results in response to questions posed to an automated information-retrieval or question-answering service
US7409335B1 (en) 2001-06-29 2008-08-05 Microsoft Corporation Inferring informational goals and preferred level of detail of answers based on application being employed by the user
US20030090437A1 (en) * 2001-11-12 2003-05-15 Adams Michael Dewayne Display system
US7644144B1 (en) 2001-12-21 2010-01-05 Microsoft Corporation Methods, tools, and interfaces for the dynamic assignment of people to groups to enable enhanced communication and collaboration
US6665712B2 (en) * 2002-01-30 2003-12-16 Hewlett-Packard Development Company, L.P. System and method for locating an output device
US8655804B2 (en) 2002-02-07 2014-02-18 Next Stage Evolution, Llc System and method for determining a characteristic of an individual
US7949713B2 (en) * 2002-02-13 2011-05-24 Panasonic Corporation Information providing apparatus, provided information presenting apparatus, and information providing method
US20030167167A1 (en) * 2002-02-26 2003-09-04 Li Gong Intelligent personal assistants
US20030163311A1 (en) * 2002-02-26 2003-08-28 Li Gong Intelligent social agents
US7203909B1 (en) 2002-04-04 2007-04-10 Microsoft Corporation System and methods for constructing personalized context-sensitive portal pages or views by analyzing patterns of users' information access activities
US7076523B2 (en) * 2002-04-24 2006-07-11 Siemens Corporate Research, Inc. Interaction interface for a composite device computing environment
US20030212761A1 (en) * 2002-05-10 2003-11-13 Microsoft Corporation Process kernel
US20030212569A1 (en) * 2002-05-10 2003-11-13 Fabio Casati System for reporting user context information
US20030217142A1 (en) 2002-05-15 2003-11-20 Microsoft Corporation Method and system for supporting the communication of presence information regarding one or more telephony devices
US20030217098A1 (en) 2002-05-15 2003-11-20 Microsoft Corporation Method and system for supporting the communication of presence information regarding one or more telephony devices
US8583440B2 (en) 2002-06-20 2013-11-12 Tegic Communications, Inc. Apparatus and method for providing visual indication of character ambiguity during text entry
US7203635B2 (en) * 2002-06-27 2007-04-10 Microsoft Corporation Layered models for context awareness
US7870240B1 (en) 2002-06-28 2011-01-11 Microsoft Corporation Metadata schema for interpersonal communications management systems
US9886309B2 (en) 2002-06-28 2018-02-06 Microsoft Technology Licensing, Llc Identity-based distributed computing for device resources
US7069259B2 (en) * 2002-06-28 2006-06-27 Microsoft Corporation Multi-attribute specification of preferences about people, priorities and privacy for guiding messaging and communications
US20070061413A1 (en) * 2005-09-15 2007-03-15 Larsen Eric J System and method for obtaining user information from voices
US20070260517A1 (en) * 2006-05-08 2007-11-08 Gary Zalewski Profile detection
US20070261077A1 (en) * 2006-05-08 2007-11-08 Gary Zalewski Using audio/visual environment to select ads on game platform
CA2397185A1 (en) * 2002-08-05 2004-02-05 Christian Cloutier Baby activity monitor
GB0218713D0 (en) * 2002-08-12 2002-09-18 Mitel Knowledge Corp Architecture and Implementation for control of context aware call processing with local feature definition
US7020508B2 (en) * 2002-08-22 2006-03-28 Bodymedia, Inc. Apparatus for detecting human physiological and contextual information
US6779904B1 (en) * 2002-08-28 2004-08-24 Rowe International Corporation Title rack lighting for jukebox
DE10247152B4 (en) * 2002-10-09 2009-05-20 Siemens Ag Remote activity controller for individuals
MXPA05003688A (en) * 2002-10-09 2005-09-30 Bodymedia Inc Method and apparatus for auto journaling of continuous or discrete body states utilizing physiological and/or contextual parameters.
US20090177068A1 (en) * 2002-10-09 2009-07-09 Stivoric John M Method and apparatus for providing derived glucose information utilizing physiological and/or contextual parameters
US20040218451A1 (en) * 2002-11-05 2004-11-04 Said Joe P. Accessible user interface and navigation system and method
US8037150B2 (en) 2002-11-21 2011-10-11 Aol Inc. System and methods for providing multiple personas in a communications environment
US7636755B2 (en) * 2002-11-21 2009-12-22 Aol Llc Multiple avatar personalities
JP2004237022A (en) * 2002-12-11 2004-08-26 Sony Corp Information processing device and method, program and recording medium
US6853955B1 (en) 2002-12-13 2005-02-08 Garmin Ltd. Portable apparatus with performance monitoring and audio entertainment features
US7480512B2 (en) 2004-01-16 2009-01-20 Bones In Motion, Inc. Wireless device, program products and methods of using a wireless device to deliver services
US20040153445A1 (en) * 2003-02-04 2004-08-05 Horvitz Eric J. Systems and methods for constructing and using models of memorability in computing and communications applications
US7536650B1 (en) 2003-02-25 2009-05-19 Robertson George G System and method that facilitates computer desktop use via scaling of displayed objects with shifts to the periphery
US8225224B1 (en) 2003-02-25 2012-07-17 Microsoft Corporation Computer desktop use via scaling of displayed objects with shifts to the periphery
US8230359B2 (en) * 2003-02-25 2012-07-24 Microsoft Corporation System and method that facilitates computer desktop use via scaling of displayed objects with shifts to the periphery
JP2006519540A (en) * 2003-02-28 2006-08-24 コーニンクレッカ フィリップス エレクトロニクス エヌ ヴィ A system to determine user preferences
US20040179037A1 (en) * 2003-03-03 2004-09-16 Blattner Patrick D. Using avatars to communicate context out-of-band
US7908554B1 (en) * 2003-03-03 2011-03-15 Aol Inc. Modifying avatar behavior based on user action or mood
US20070113181A1 (en) * 2003-03-03 2007-05-17 Blattner Patrick D Using avatars to communicate real-time information
US7913176B1 (en) 2003-03-03 2011-03-22 Aol Inc. Applying access controls to communications with avatars
US7793233B1 (en) 2003-03-12 2010-09-07 Microsoft Corporation System and method for customizing note flags
US7026928B1 (en) 2003-03-21 2006-04-11 Realty Times Portable personal security system
US7774799B1 (en) 2003-03-26 2010-08-10 Microsoft Corporation System and method for linking page content with a media file and displaying the links
US7457879B2 (en) * 2003-04-01 2008-11-25 Microsoft Corporation Notification platform architecture
EP1629359A4 (en) 2003-04-07 2008-01-09 Sevenecho Llc Method, system and software for digital media narrative personalization
US7182738B2 (en) 2003-04-23 2007-02-27 Marctec, Llc Patient monitoring apparatus and method for orthosis and other devices
US6992625B1 (en) * 2003-04-25 2006-01-31 Microsoft Corporation Calibration of a device location measurement system that utilizes wireless signal strengths
US7729711B2 (en) * 2003-05-09 2010-06-01 Intel Corporation Reducing interference from closely proximate wireless units
US20050007303A1 (en) * 2003-06-13 2005-01-13 Ernest Betton Small screen larger than life 3D TV
US6837827B1 (en) 2003-06-17 2005-01-04 Garmin Ltd. Personal training device using GPS data
US8339462B2 (en) 2008-01-28 2012-12-25 DigitalOptics Corporation Europe Limited Methods and apparatuses for addressing chromatic abberations and purple fringing
US7162473B2 (en) * 2003-06-26 2007-01-09 Microsoft Corporation Method and system for usage analyzer that determines user accessed sources, indexes data subsets, and associated metadata, processing implicit queries based on potential interest to users
US20040267746A1 (en) * 2003-06-26 2004-12-30 Cezary Marcjan User interface for controlling access to computer objects
US7225187B2 (en) * 2003-06-26 2007-05-29 Microsoft Corporation Systems and methods for performing background queries from content and activity
US8346587B2 (en) 2003-06-30 2013-01-01 Microsoft Corporation Models and methods for reducing visual complexity and search effort via ideal information abstraction, hiding, and sequencing
US7444598B2 (en) * 2003-06-30 2008-10-28 Microsoft Corporation Exploded views for providing rich regularized geometric transformations and interaction models on content for viewing, previewing, and interacting with documents, projects, and tasks
US20040264677A1 (en) * 2003-06-30 2004-12-30 Horvitz Eric J. Ideal transfer of call handling from automated systems to human operators based on forecasts of automation efficacy and operator load
US7250907B2 (en) * 2003-06-30 2007-07-31 Microsoft Corporation System and methods for determining the location dynamics of a portable computing device
US20050108366A1 (en) * 2003-07-02 2005-05-19 International Business Machines Corporation Administering devices with domain state objects
US7738881B2 (en) * 2003-07-22 2010-06-15 Microsoft Corporation Systems for determining the approximate location of a device from ambient signals
US7319877B2 (en) * 2003-07-22 2008-01-15 Microsoft Corporation Methods for determining the approximate location of a device from ambient signals
US7202816B2 (en) * 2003-07-22 2007-04-10 Microsoft Corporation Utilization of the approximate location of a device determined from ambient signals
US7454393B2 (en) * 2003-08-06 2008-11-18 Microsoft Corporation Cost-benefit approach to automatically composing answers to questions by extracting information from large unstructured corpora
KR101157016B1 (en) * 2003-08-06 2012-06-21 코닌클리케 필립스 일렉트로닉스 엔.브이. A method of presenting a plurality of items
US7529674B2 (en) * 2003-08-18 2009-05-05 Sap Aktiengesellschaft Speech animation
CA2538710A1 (en) * 2003-09-12 2005-03-31 Bodymedia, Inc. Method and apparatus for measuring heart related parameters
US20050084082A1 (en) * 2003-10-15 2005-04-21 Microsoft Corporation Designs, interfaces, and policies for systems that enhance communication and minimize disruption by encoding preferences and situations
US7831679B2 (en) * 2003-10-15 2010-11-09 Microsoft Corporation Guiding sensing and preferences for context-sensitive services
EP1524586A1 (en) 2003-10-17 2005-04-20 Sony International (Europe) GmbH Transmitting information to a user's body
US7774349B2 (en) * 2003-12-11 2010-08-10 Microsoft Corporation Statistical models and methods to support the personalization of applications and services via consideration of preference encodings of a community of users
US7427024B1 (en) 2003-12-17 2008-09-23 Gazdzinski Mark J Chattel management apparatus and methods
US6882897B1 (en) * 2004-01-05 2005-04-19 Dennis S. Fernandez Reconfigurable garment definition and production method
US7401300B2 (en) * 2004-01-09 2008-07-15 Nokia Corporation Adaptive user interface input device
US20050165631A1 (en) * 2004-01-28 2005-07-28 Microsoft Corporation Time management representations and automation for allocating time to projects and meetings within an online calendaring system
US7398151B1 (en) 2004-02-25 2008-07-08 Garmin Ltd. Wearable electronic device
US7293019B2 (en) * 2004-03-02 2007-11-06 Microsoft Corporation Principles and methods for personalizing newsfeeds via an analysis of information novelty and dynamics
US7327349B2 (en) * 2004-03-02 2008-02-05 Microsoft Corporation Advanced navigation techniques for portable devices
EP1734858B1 (en) 2004-03-22 2014-07-09 BodyMedia, Inc. Non-invasive temperature monitoring device
US7908663B2 (en) 2004-04-20 2011-03-15 Microsoft Corporation Abstractions and automation for enhanced sharing and collaboration
US7057551B1 (en) 2004-04-27 2006-06-06 Garmin Ltd. Electronic exercise monitor and method using a location determining component and a pedometer
US20070203589A1 (en) * 2005-04-08 2007-08-30 Manyworlds, Inc. Adaptive Recombinant Process Methods
WO2005119356A2 (en) 2004-05-28 2005-12-15 Erik Jan Banning Interactive direct-pointing system and calibration method
JP2005339377A (en) * 2004-05-28 2005-12-08 Canon Inc Image processing method and image processor
US8095364B2 (en) 2004-06-02 2012-01-10 Tegic Communications, Inc. Multimodal disambiguation of speech recognition
US20070192711A1 (en) * 2006-02-13 2007-08-16 Research In Motion Limited Method and arrangement for providing a primary actions menu on a handheld communication device
US8463315B2 (en) 2004-06-21 2013-06-11 Research In Motion Limited Handheld wireless communication device
US7664249B2 (en) * 2004-06-30 2010-02-16 Microsoft Corporation Methods and interfaces for probing and understanding behaviors of alerting and filtering systems based on models and simulation from logs
US20060012183A1 (en) * 2004-07-19 2006-01-19 David Marchiori Rail car door opener
US20060259866A1 (en) * 2004-07-20 2006-11-16 Balaji Prasad Virtual user interface for multiple user devices
US7599838B2 (en) * 2004-09-01 2009-10-06 Sap Aktiengesellschaft Speech animation with behavioral contexts for application scenarios
JP4360308B2 (en) * 2004-09-21 2009-11-11 株式会社デンソー In-vehicle acoustic control system and AI agent
US7480567B2 (en) * 2004-09-24 2009-01-20 Nokia Corporation Displaying a map having a close known location
US20060070001A1 (en) * 2004-09-29 2006-03-30 Fuji Xerox Co., Ltd. Computer assisted presentation authoring for multimedia venues
US7788589B2 (en) 2004-09-30 2010-08-31 Microsoft Corporation Method and system for improved electronic task flagging and management
US7712049B2 (en) 2004-09-30 2010-05-04 Microsoft Corporation Two-dimensional radial user interface for computer software applications
US20060074883A1 (en) * 2004-10-05 2006-04-06 Microsoft Corporation Systems, methods, and interfaces for providing personalized search and information access
US7519564B2 (en) * 2004-11-16 2009-04-14 Microsoft Corporation Building and using predictive models of current and future surprises
US7610560B2 (en) 2004-11-16 2009-10-27 Microsoft Corporation Methods for automated and semiautomated composition of visual sequences, flows, and flyovers based on content and context
US7698055B2 (en) * 2004-11-16 2010-04-13 Microsoft Corporation Traffic forecasting employing modeling and analysis of probabilistic interdependencies and contextual data
US7327245B2 (en) * 2004-11-22 2008-02-05 Microsoft Corporation Sensing and analysis of ambient contextual signals for discriminating between indoor and outdoor locations
US20060135139A1 (en) * 2004-12-17 2006-06-22 Cheng Steven D Method for changing outputting settings for a mobile unit based on user's physical status
US9652809B1 (en) 2004-12-21 2017-05-16 Aol Inc. Using user profile information to determine an avatar and/or avatar characteristics
US8375434B2 (en) * 2004-12-31 2013-02-12 Ntrepid Corporation System for protecting identity in a network environment
US7356473B2 (en) * 2005-01-21 2008-04-08 Lawrence Kates Management and assistance system for the deaf
US20060192775A1 (en) * 2005-02-25 2006-08-31 Microsoft Corporation Using detected visual cues to change computer system operating states
US7689615B2 (en) * 2005-02-25 2010-03-30 Microsoft Corporation Ranking results using multiple nested ranking
US7885817B2 (en) 2005-03-08 2011-02-08 Microsoft Corporation Easy generation and automatic training of spoken dialog systems using text-to-speech
US7707131B2 (en) * 2005-03-08 2010-04-27 Microsoft Corporation Thompson strategy based online reinforcement learning system for action selection
US7734471B2 (en) * 2005-03-08 2010-06-08 Microsoft Corporation Online learning for dialog systems
US20060206333A1 (en) * 2005-03-08 2006-09-14 Microsoft Corporation Speaker-dependent dialog adaptation
CN1855095A (en) * 2005-04-27 2006-11-01 国际商业机器公司 System, method and engine for playing multimedia content based on SMIL
US20060260624A1 (en) * 2005-05-17 2006-11-23 Battelle Memorial Institute Method, program, and system for automatic profiling of entities
US20070011109A1 (en) * 2005-06-23 2007-01-11 Microsoft Corporation Immortal information storage and access platform
US7991607B2 (en) * 2005-06-27 2011-08-02 Microsoft Corporation Translation and capture architecture for output of conversational utterances
US7643985B2 (en) * 2005-06-27 2010-01-05 Microsoft Corporation Context-sensitive communication and translation methods for enhanced interactions and understanding among speakers of different languages
US7647171B2 (en) * 2005-06-29 2010-01-12 Microsoft Corporation Learning, storing, analyzing, and reasoning about the loss of location-identifying signals
US8079079B2 (en) * 2005-06-29 2011-12-13 Microsoft Corporation Multimodal authentication
US20070005363A1 (en) * 2005-06-29 2007-01-04 Microsoft Corporation Location aware multi-modal multi-lingual device
US7693817B2 (en) * 2005-06-29 2010-04-06 Microsoft Corporation Sensing, storing, indexing, and retrieving data leveraging measures of user activity, attention, and interest
US7529683B2 (en) * 2005-06-29 2009-05-05 Microsoft Corporation Principals and methods for balancing the timeliness of communications and information delivery with the expected cost of interruption via deferral policies
US7694214B2 (en) * 2005-06-29 2010-04-06 Microsoft Corporation Multimodal note taking, annotation, and gaming
US7428521B2 (en) * 2005-06-29 2008-09-23 Microsoft Corporation Precomputation of context-sensitive policies for automated inquiry and action under uncertainty
US7460884B2 (en) * 2005-06-29 2008-12-02 Microsoft Corporation Data buddy
US20070004969A1 (en) * 2005-06-29 2007-01-04 Microsoft Corporation Health monitor
US7646755B2 (en) * 2005-06-30 2010-01-12 Microsoft Corporation Seamless integration of portable computing devices and desktop computers
US7925995B2 (en) * 2005-06-30 2011-04-12 Microsoft Corporation Integration of location logs, GPS signals, and spatial resources for identifying user activities, goals, and context
US20070005646A1 (en) * 2005-06-30 2007-01-04 Microsoft Corporation Analysis of topic dynamics of web search
US20070005754A1 (en) * 2005-06-30 2007-01-04 Microsoft Corporation Systems and methods for triaging attention for providing awareness of communications session activity
US20070010323A1 (en) * 2005-07-05 2007-01-11 International Business Machines Corporation Platform-adaptive gaming
US20070165019A1 (en) * 2005-07-12 2007-07-19 Hale Kelly S Design Of systems For Improved Human Interaction
US9285897B2 (en) 2005-07-13 2016-03-15 Ultimate Pointer, L.L.C. Easily deployable interactive direct-pointing system and calibration method therefor
EP1917571A2 (en) * 2005-08-15 2008-05-07 Philips Intellectual Property & Standards GmbH User interface system for a personal healthcare environment
JP2007053641A (en) * 2005-08-19 2007-03-01 Sony Corp Communication controller and communication control system
US20070050251A1 (en) * 2005-08-29 2007-03-01 Microsoft Corporation Monetizing a preview pane for ads
US20070050253A1 (en) * 2005-08-29 2007-03-01 Microsoft Corporation Automatically generating content for presenting in a preview pane for ADS
US20070050252A1 (en) * 2005-08-29 2007-03-01 Microsoft Corporation Preview pane for ads
US8645985B2 (en) 2005-09-15 2014-02-04 Sony Computer Entertainment Inc. System and method for detecting user attention
US8616973B2 (en) * 2005-09-15 2013-12-31 Sony Computer Entertainment Inc. System and method for control by audible device
US20070106628A1 (en) * 2005-09-29 2007-05-10 Conopco, Inc., D/B/A Unilever Dialogue strategies
US8024112B2 (en) * 2005-09-29 2011-09-20 Microsoft Corporation Methods for predicting destinations from partial trajectories employing open-and closed-world modeling methods
US7319908B2 (en) * 2005-10-28 2008-01-15 Microsoft Corporation Multi-modal device power/mode management
US20070100704A1 (en) * 2005-10-28 2007-05-03 Microsoft Corporation Shopping assistant
US7778632B2 (en) * 2005-10-28 2010-08-17 Microsoft Corporation Multi-modal device capable of automated actions
US7467353B2 (en) * 2005-10-28 2008-12-16 Microsoft Corporation Aggregation of multi-modal devices
US7636794B2 (en) * 2005-10-31 2009-12-22 Microsoft Corporation Distributed sensing techniques for mobile devices
US20070101287A1 (en) * 2005-11-03 2007-05-03 International Business Machines Corporation Pop-up windows in a computer system
US20070112906A1 (en) * 2005-11-15 2007-05-17 Microsoft Corporation Infrastructure for multi-modal multilingual communications devices
US20070117072A1 (en) * 2005-11-21 2007-05-24 Conopco Inc, D/B/A Unilever Attitude reaction monitoring
US20070136222A1 (en) * 2005-12-09 2007-06-14 Microsoft Corporation Question and answer architecture for reasoning and clarifying intentions, goals, and needs from contextual clues and content
US20070136068A1 (en) * 2005-12-09 2007-06-14 Microsoft Corporation Multimodal multilingual devices and applications for enhanced goal-interpretation and translation for service providers
US20070150512A1 (en) * 2005-12-15 2007-06-28 Microsoft Corporation Collaborative meeting assistant
US7774851B2 (en) * 2005-12-22 2010-08-10 Scenera Technologies, Llc Methods, systems, and computer program products for protecting information on a user interface based on a viewability of the information
US7747557B2 (en) 2006-01-05 2010-06-29 Microsoft Corporation Application of metadata to documents and document objects via an operating system user interface
US7797638B2 (en) 2006-01-05 2010-09-14 Microsoft Corporation Application of metadata to documents and document objects via a software application user interface
US8020993B1 (en) 2006-01-30 2011-09-20 Fram Evan K Viewing verification systems
US7817991B2 (en) * 2006-02-14 2010-10-19 Microsoft Corporation Dynamic interconnection of mobile devices
US20070197881A1 (en) * 2006-02-22 2007-08-23 Wolf James L Wireless Health Monitor Device and System with Cognition
US20070225577A1 (en) * 2006-03-01 2007-09-27 Honeywell International Inc. System and Method for Providing Sensor Based Human Factors Protocol Analysis
US7617164B2 (en) * 2006-03-17 2009-11-10 Microsoft Corporation Efficiency of training for ranking systems based on pairwise training with aggregated gradients
US20070226033A1 (en) * 2006-03-21 2007-09-27 Lopresti Edmund F Distributed cognitive aid method and system for scheduling and interactive task guidance
US20070243930A1 (en) * 2006-04-12 2007-10-18 Gary Zalewski System and method for using user's audio environment to select advertising
US20070244751A1 (en) * 2006-04-17 2007-10-18 Gary Zalewski Using visual environment to select ads on game platform
US20070255630A1 (en) * 2006-04-17 2007-11-01 Gary Zalewski System and method for using user's visual environment to select advertising
DE102006017509B4 (en) * 2006-04-13 2008-08-14 Maxie Pantel Device for translating sign language
US9386327B2 (en) 2006-05-24 2016-07-05 Time Warner Cable Enterprises Llc Secondary content insertion apparatus and methods
US8280982B2 (en) 2006-05-24 2012-10-02 Time Warner Cable Inc. Personal content server apparatus and methods
US7920071B2 (en) * 2006-05-26 2011-04-05 Itt Manufacturing Enterprises, Inc. Augmented reality-based system and method providing status and control of unmanned vehicles
US9323055B2 (en) * 2006-05-26 2016-04-26 Exelis, Inc. System and method to display maintenance and operational instructions of an apparatus using augmented reality
US20070282783A1 (en) * 2006-05-31 2007-12-06 Mona Singh Automatically determining a sensitivity level of a resource and applying presentation attributes to the resource based on attributes of a user environment
US8024762B2 (en) 2006-06-13 2011-09-20 Time Warner Cable Inc. Methods and apparatus for providing virtual content over a network
US7761464B2 (en) * 2006-06-19 2010-07-20 Microsoft Corporation Diversifying search results for improved search and personalization
CN101472518B (en) * 2006-06-20 2011-02-16 夏普株式会社 Setting device, biometric device setting system, biometric device setting method
US8099086B2 (en) * 2006-06-21 2012-01-17 Ektimisi Semiotics Holdings, Llc System and method for providing a descriptor for a location to a recipient
US8750892B2 (en) 2006-06-21 2014-06-10 Scenera Mobile Technologies, Llc System and method for naming a location based on user-specific information
US7610151B2 (en) 2006-06-27 2009-10-27 Microsoft Corporation Collaborative route planning for generating personalized and context-sensitive routing recommendations
US7917514B2 (en) * 2006-06-28 2011-03-29 Microsoft Corporation Visual and multi-dimensional search
US20080004948A1 (en) * 2006-06-28 2008-01-03 Microsoft Corporation Auctioning for video and audio advertising
US20080005069A1 (en) * 2006-06-28 2008-01-03 Microsoft Corporation Entity-specific search model
US9396269B2 (en) * 2006-06-28 2016-07-19 Microsoft Technology Licensing, Llc Search engine that identifies and uses social networks in communications, retrieval, and electronic commerce
US7739221B2 (en) * 2006-06-28 2010-06-15 Microsoft Corporation Visual and multi-dimensional search
US20080004990A1 (en) * 2006-06-28 2008-01-03 Microsoft Corporation Virtual spot market for advertisements
US8874592B2 (en) 2006-06-28 2014-10-28 Microsoft Corporation Search guided by location and context
US20080005068A1 (en) * 2006-06-28 2008-01-03 Microsoft Corporation Context-based search, retrieval, and awareness
US9141704B2 (en) * 2006-06-28 2015-09-22 Microsoft Technology Licensing, Llc Data management in social networks
US20080005067A1 (en) * 2006-06-28 2008-01-03 Microsoft Corporation Context-based search, retrieval, and awareness
US20080005223A1 (en) * 2006-06-28 2008-01-03 Microsoft Corporation Reputation data for entities and data processing
US20080005104A1 (en) * 2006-06-28 2008-01-03 Microsoft Corporation Localized marketing
US20080005074A1 (en) * 2006-06-28 2008-01-03 Microsoft Corporation Search over designated content
US20080005108A1 (en) * 2006-06-28 2008-01-03 Microsoft Corporation Message mining to enhance ranking of documents for retrieval
US7822762B2 (en) * 2006-06-28 2010-10-26 Microsoft Corporation Entity-specific search model
US7984169B2 (en) * 2006-06-28 2011-07-19 Microsoft Corporation Anonymous and secure network-based interaction
US20080005095A1 (en) * 2006-06-28 2008-01-03 Microsoft Corporation Validation of computer responses
US8788517B2 (en) * 2006-06-28 2014-07-22 Microsoft Corporation Intelligently guiding search based on user dialog
US20080004884A1 (en) * 2006-06-29 2008-01-03 Microsoft Corporation Employment of offline behavior to display online content
US20080004951A1 (en) * 2006-06-29 2008-01-03 Microsoft Corporation Web-based targeted advertising in a brick-and-mortar retail establishment using online customer information
US20080005047A1 (en) * 2006-06-29 2008-01-03 Microsoft Corporation Scenario-based search
US7552862B2 (en) * 2006-06-29 2009-06-30 Microsoft Corporation User-controlled profile sharing
US7873620B2 (en) * 2006-06-29 2011-01-18 Microsoft Corporation Desktop search from mobile device
US8626136B2 (en) * 2006-06-29 2014-01-07 Microsoft Corporation Architecture for user- and context-specific prefetching and caching of information on portable devices
US8725567B2 (en) * 2006-06-29 2014-05-13 Microsoft Corporation Targeted advertising in brick-and-mortar establishments
US20080005079A1 (en) * 2006-06-29 2008-01-03 Microsoft Corporation Scenario-based search
US7997485B2 (en) 2006-06-29 2011-08-16 Microsoft Corporation Content presentation based on user preferences
US8244240B2 (en) * 2006-06-29 2012-08-14 Microsoft Corporation Queries as data for revising and extending a sensor-based location service
US20080005313A1 (en) * 2006-06-29 2008-01-03 Microsoft Corporation Using offline activity to enhance online searching
US8126641B2 (en) * 2006-06-30 2012-02-28 Microsoft Corporation Route planning with contingencies
US7739040B2 (en) 2006-06-30 2010-06-15 Microsoft Corporation Computation of travel routes, durations, and plans over multiple contexts
US20080004954A1 (en) * 2006-06-30 2008-01-03 Microsoft Corporation Methods and architecture for performing client-side directed marketing with caching and local analytics for enhanced privacy and minimal disruption
US8112755B2 (en) * 2006-06-30 2012-02-07 Microsoft Corporation Reducing latencies in computing systems using probabilistic and/or decision-theoretic reasoning under scarce memory resources
US7617042B2 (en) * 2006-06-30 2009-11-10 Microsoft Corporation Computing and harnessing inferences about the timing, duration, and nature of motion and cessation of motion with applications to mobile computing and communications
US7706964B2 (en) * 2006-06-30 2010-04-27 Microsoft Corporation Inferring road speeds for context-sensitive routing
US7797267B2 (en) * 2006-06-30 2010-09-14 Microsoft Corporation Methods and architecture for learning and reasoning in support of context-sensitive reminding, informing, and service facilitation
US20080034435A1 (en) * 2006-08-03 2008-02-07 Ibm Corporation Methods and arrangements for detecting and managing viewability of screens, windows and like media
US8407213B2 (en) 2006-08-31 2013-03-26 Ektimisi Semiotics Holdings, Llc System and method for identifying a location of interest to be named by a user
US7649454B2 (en) 2006-09-28 2010-01-19 Ektimisi Semiotics Holdings, Llc System and method for providing a task reminder based on historical travel information
US7528713B2 (en) 2006-09-28 2009-05-05 Ektimisi Semiotics Holdings, Llc Apparatus and method for providing a task reminder based on travel history
US9396185B2 (en) * 2006-10-31 2016-07-19 Scenera Mobile Technologies, Llc Method and apparatus for providing a contextual description of an object
US7707518B2 (en) 2006-11-13 2010-04-27 Microsoft Corporation Linking information
US7761785B2 (en) 2006-11-13 2010-07-20 Microsoft Corporation Providing resilient links
US20080154720A1 (en) * 2006-12-20 2008-06-26 Microsoft Corporation Shopping route optimization and personalization
US20080153513A1 (en) * 2006-12-20 2008-06-26 Microsoft Corporation Mobile ad selection and filtering
US8805720B2 (en) * 2006-12-20 2014-08-12 Microsoft Corporation Feedback loop for consumer transactions
US20080154703A1 (en) * 2006-12-20 2008-06-26 Microsoft Corporation Retailer competition based on published intent
US7765173B2 (en) * 2006-12-31 2010-07-27 Ektimisi Semiotics Holdings, Llc Method, system, and computer program product for delivering smart services
US8099084B2 (en) 2006-12-31 2012-01-17 Ektimisi Semiotics Holdings, Llc Method, system, and computer program product for creating smart services
US7647283B2 (en) * 2006-12-31 2010-01-12 Ektimisi Semiotics Holdings, Llc Method, system, and computer program product for adaptively learning user preferences for smart services
EP2126828A4 (en) * 2007-02-16 2012-01-25 Bodymedia Inc Systems and methods for understanding and applying the physiological and contextual life patterns of an individual or set of individuals
US8181206B2 (en) 2007-02-28 2012-05-15 Time Warner Cable Inc. Personal content server apparatus and methods
US7711716B2 (en) * 2007-03-06 2010-05-04 Microsoft Corporation Optimizations for a background database consistency check
US20080218331A1 (en) * 2007-03-08 2008-09-11 Itt Manufacturing Enterprises, Inc. Augmented reality-based system and method to show the location of personnel and sensors inside occluded structures and provide increased situation awareness
US8032472B2 (en) * 2007-04-04 2011-10-04 Tuen Solutions Limited Liability Company Intelligent agent for distributed services for mobile devices
JP4938530B2 (en) * 2007-04-06 2012-05-23 株式会社エヌ・ティ・ティ・ドコモ Mobile communication terminal and program
US20080249667A1 (en) * 2007-04-09 2008-10-09 Microsoft Corporation Learning and reasoning to enhance energy efficiency in transportation systems
US20080256495A1 (en) * 2007-04-10 2008-10-16 Microsoft Corporation Personalized user interface
US8621348B2 (en) * 2007-05-25 2013-12-31 Immersion Corporation Customizing haptic effects on an end user device
US7539659B2 (en) * 2007-06-15 2009-05-26 Microsoft Corporation Multidimensional timeline browsers for broadcast media
US7970721B2 (en) * 2007-06-15 2011-06-28 Microsoft Corporation Learning and reasoning from web projections
US9754078B2 (en) * 2007-06-21 2017-09-05 Immersion Corporation Haptic health feedback monitoring
US7979252B2 (en) * 2007-06-21 2011-07-12 Microsoft Corporation Selective sampling of user state based on expected utility
US20080320087A1 (en) * 2007-06-22 2008-12-25 Microsoft Corporation Swarm sensing and actuating
US7912637B2 (en) * 2007-06-25 2011-03-22 Microsoft Corporation Landmark-based routing
US20080319658A1 (en) * 2007-06-25 2008-12-25 Microsoft Corporation Landmark-based routing
US20080319660A1 (en) * 2007-06-25 2008-12-25 Microsoft Corporation Landmark-based routing
US7991718B2 (en) * 2007-06-28 2011-08-02 Microsoft Corporation Method and apparatus for generating an inference about a destination of a trip using a combination of open-world modeling and closed world modeling
US7696866B2 (en) * 2007-06-28 2010-04-13 Microsoft Corporation Learning and reasoning about the context-sensitive reliability of sensors
US8244660B2 (en) 2007-06-28 2012-08-14 Microsoft Corporation Open-world modeling
US8254393B2 (en) * 2007-06-29 2012-08-28 Microsoft Corporation Harnessing predictive models of durations of channel availability for enhanced opportunistic allocation of radio spectrum
US7948400B2 (en) * 2007-06-29 2011-05-24 Microsoft Corporation Predictive models of road reliability for traffic sensor configuration and routing
US7673088B2 (en) * 2007-06-29 2010-03-02 Microsoft Corporation Multi-tasking interference model
US20090002178A1 (en) * 2007-06-29 2009-01-01 Microsoft Corporation Dynamic mood sensing
ITUD20070127A1 (en) * 2007-07-16 2009-01-17 Eurotech S P A WEARABLE ELECTRONIC EQUIPMENT
US8081158B2 (en) * 2007-08-06 2011-12-20 Harris Technology, Llc Intelligent display screen which interactively selects content to be displayed based on surroundings
JP5082722B2 (en) * 2007-09-28 2012-11-28 ブラザー工業株式会社 Image display device and image display system
EP2053579A3 (en) * 2007-10-24 2012-08-08 Brother Kogyo Kabushiki Kaisha Data processing device
US9513699B2 (en) * 2007-10-24 2016-12-06 Invention Science Fund I, LL Method of selecting a second content based on a user's reaction to a first content
US9582805B2 (en) 2007-10-24 2017-02-28 Invention Science Fund I, Llc Returning a personalized advertisement
JP4935658B2 (en) * 2007-12-11 2012-05-23 ブラザー工業株式会社 Browser program and information processing apparatus
US20090158190A1 (en) * 2007-12-13 2009-06-18 Yuvee, Inc. Computing apparatus including a personal web and application assistant
US20090155753A1 (en) * 2007-12-14 2009-06-18 Kimberly-Clark Worldwide, Inc. Behavior Tracking with Tracking Pods
US20090177601A1 (en) * 2008-01-08 2009-07-09 Microsoft Corporation Status-aware personal information management
JP5061931B2 (en) * 2008-02-04 2012-10-31 ソニー株式会社 Information processing apparatus and information processing method
US9503691B2 (en) 2008-02-19 2016-11-22 Time Warner Cable Enterprises Llc Methods and apparatus for enhanced advertising and promotional delivery in a network
US9161715B2 (en) * 2008-05-23 2015-10-20 Invention Science Fund I, Llc Determination of extent of congruity between observation of authoring user and observation of receiving user
US8615664B2 (en) * 2008-05-23 2013-12-24 The Invention Science Fund I, Llc Acquisition and particular association of inference data indicative of an inferred mental state of an authoring user and source identity data
US9192300B2 (en) * 2008-05-23 2015-11-24 Invention Science Fund I, Llc Acquisition and particular association of data indicative of an inferred mental state of an authoring user
US20090292658A1 (en) * 2008-05-23 2009-11-26 Searete Llc, A Limited Liability Corporation Of The State Of Delaware Acquisition and particular association of inference data indicative of inferred mental states of authoring users
US9101263B2 (en) * 2008-05-23 2015-08-11 The Invention Science Fund I, Llc Acquisition and association of data indicative of an inferred mental state of an authoring user
US9357052B2 (en) * 2008-06-09 2016-05-31 Immersion Corporation Developing a notification framework for electronic device events
US9225817B2 (en) * 2008-06-16 2015-12-29 Sony Corporation Method and apparatus for providing motion activated updating of weather information
US8516001B2 (en) * 2008-06-24 2013-08-20 Microsoft Corporation Context platform
US20090320143A1 (en) * 2008-06-24 2009-12-24 Microsoft Corporation Sensor interface
US9846049B2 (en) * 2008-07-09 2017-12-19 Microsoft Technology Licensing, Llc Route prediction
US8712958B2 (en) * 2008-07-16 2014-04-29 Electronics And Telecommunications Research Institute Method and apparatus for representing sensory effects and computer readable recording medium storing user sensory preference metadata
US20100076334A1 (en) 2008-09-19 2010-03-25 Unither Neurosciences, Inc. Alzheimer's cognitive enabler
US9409052B2 (en) 2008-10-03 2016-08-09 Adidas Ag Program products, methods, and systems for providing location-aware fitness monitoring services
US20100088143A1 (en) * 2008-10-07 2010-04-08 Microsoft Corporation Calendar event scheduling
US9189256B2 (en) * 2008-11-20 2015-11-17 Nokia Technologies Oy Method and apparatus for utilizing user identity
US10489053B2 (en) * 2008-11-20 2019-11-26 Gula Consulting Limited Liability Company Method and apparatus for associating user identity
KR101173944B1 (en) * 2008-12-01 2012-08-20 한국전자통신연구원 System and method for controlling sensibility of driver
US8494857B2 (en) 2009-01-06 2013-07-23 Regents Of The University Of Minnesota Automatic measurement of speech fluency
US9596989B2 (en) * 2009-03-12 2017-03-21 Raytheon Company Networked symbiotic edge user infrastructure
US8346800B2 (en) * 2009-04-02 2013-01-01 Microsoft Corporation Content-based information retrieval
US8661030B2 (en) 2009-04-09 2014-02-25 Microsoft Corporation Re-ranking top search results
US20100292545A1 (en) * 2009-05-14 2010-11-18 Advanced Brain Monitoring, Inc. Interactive psychophysiological profiler method and system
US20100295782A1 (en) 2009-05-21 2010-11-25 Yehuda Binder System and method for control based on face ore hand gesture detection
US20100313133A1 (en) * 2009-06-08 2010-12-09 Microsoft Corporation Audio and position control of user interface
JP5035303B2 (en) * 2009-06-17 2012-09-26 トヨタ自動車株式会社 MOBILE BODY, SYSTEM INCLUDING THE SAME, MOBILE BODY OPERATING METHOD, AND PROGRAM
US8427508B2 (en) * 2009-06-25 2013-04-23 Nokia Corporation Method and apparatus for an augmented reality user interface
KR101584058B1 (en) * 2009-08-07 2016-01-12 삼성전자주식회사 Mobile terminal providing environment adapted to present situation and method for operating the same
KR101544371B1 (en) * 2009-08-07 2015-08-17 삼성전자주식회사 Mobile terminal reflecting user's environment and method for operating the same
US8121618B2 (en) 2009-10-28 2012-02-21 Digimarc Corporation Intuitive computing methods and systems
KR101262922B1 (en) * 2009-12-10 2013-05-09 한국전자통신연구원 Apparatus and method for determining emotional quotient according to emotion variation
US11117033B2 (en) 2010-04-26 2021-09-14 Wilbert Quinc Murdock Smart system for display of dynamic movement parameters in sports and training
US9183560B2 (en) 2010-05-28 2015-11-10 Daniel H. Abelow Reality alternate
US8560365B2 (en) 2010-06-08 2013-10-15 International Business Machines Corporation Probabilistic optimization of resource discovery, reservation and assignment
US9305263B2 (en) 2010-06-30 2016-04-05 Microsoft Technology Licensing, Llc Combining human and machine intelligence to solve tasks with crowd sourcing
US10398366B2 (en) 2010-07-01 2019-09-03 Nokia Technologies Oy Responding to changes in emotional condition of a user
US20120011477A1 (en) * 2010-07-12 2012-01-12 Nokia Corporation User interfaces
US10496714B2 (en) * 2010-08-06 2019-12-03 Google Llc State-dependent query response
US9646271B2 (en) 2010-08-06 2017-05-09 International Business Machines Corporation Generating candidate inclusion/exclusion cohorts for a multiply constrained group
US8968197B2 (en) * 2010-09-03 2015-03-03 International Business Machines Corporation Directing a user to a medical resource
US9292577B2 (en) 2010-09-17 2016-03-22 International Business Machines Corporation User accessibility to data analytics
US9014911B2 (en) * 2011-11-16 2015-04-21 Flextronics Ap, Llc Street side sensors
US20120109868A1 (en) * 2010-11-01 2012-05-03 Microsoft Corporation Real-Time Adaptive Output
US8640021B2 (en) 2010-11-12 2014-01-28 Microsoft Corporation Audience-based presentation and customization of content
GB2531971B (en) * 2010-11-15 2016-08-17 Edesix Ltd Image recording apparatus
US8565783B2 (en) 2010-11-24 2013-10-22 Microsoft Corporation Path progression matching for indoor positioning systems
US9589254B2 (en) 2010-12-08 2017-03-07 Microsoft Technology Licensing, Llc Using e-mail message characteristics for prioritization
US9134137B2 (en) 2010-12-17 2015-09-15 Microsoft Technology Licensing, Llc Mobile search based on predicted location
TWI422504B (en) * 2010-12-31 2014-01-11 Altek Corp Vehicle apparatus control system and method thereof
TWI546700B (en) * 2011-01-13 2016-08-21 宏達國際電子股份有限公司 Portable electronic device, and control method and computer program product of the same
US9020419B2 (en) 2011-01-14 2015-04-28 Covidien, LP Wireless relay module for remote monitoring systems having power and medical device proximity monitoring functionality
US9495511B2 (en) 2011-03-01 2016-11-15 Covidien Lp Remote monitoring systems and methods for medical devices
US20130311915A1 (en) * 2011-01-27 2013-11-21 Nec Corporation Ui creation support system, ui creation support method, and non-transitory storage medium
WO2012128952A2 (en) * 2011-03-18 2012-09-27 Battelle Memorial Institute Apparatuses and methods of determining if a person operating equipment is experiencing an elevated cognitive load
US9163952B2 (en) 2011-04-15 2015-10-20 Microsoft Technology Licensing, Llc Suggestive mapping
US8620113B2 (en) 2011-04-25 2013-12-31 Microsoft Corporation Laser diode modes
US8760395B2 (en) 2011-05-31 2014-06-24 Microsoft Corporation Gesture recognition techniques
US8981995B2 (en) 2011-06-03 2015-03-17 Microsoft Technology Licensing, Llc. Low accuracy positional data by detecting improbable samples
US20120316455A1 (en) * 2011-06-10 2012-12-13 Aliphcom Wearable device and platform for sensory input
CN104011712B (en) 2011-06-24 2018-04-24 谷歌有限责任公司 To being evaluated across the query translation of language inquiry suggestion
US20130018659A1 (en) * 2011-07-12 2013-01-17 Google Inc. Systems and Methods for Speech Command Processing
US9464903B2 (en) 2011-07-14 2016-10-11 Microsoft Technology Licensing, Llc Crowd sourcing based on dead reckoning
US9470529B2 (en) 2011-07-14 2016-10-18 Microsoft Technology Licensing, Llc Activating and deactivating sensors for dead reckoning
US8538686B2 (en) 2011-09-09 2013-09-17 Microsoft Corporation Transport-dependent prediction of destinations
CN103918247B (en) 2011-09-23 2016-08-24 数字标记公司 Intelligent mobile phone sensor logic based on background environment
US9286711B2 (en) 2011-09-30 2016-03-15 Microsoft Technology Licensing, Llc Representing a location at a previous time period using an augmented reality display
US20130083008A1 (en) * 2011-09-30 2013-04-04 Kevin A. Geisner Enriched experience using personal a/v system
US20130083007A1 (en) * 2011-09-30 2013-04-04 Kevin A. Geisner Changing experience using personal a/v system
US9345957B2 (en) 2011-09-30 2016-05-24 Microsoft Technology Licensing, Llc Enhancing a sport using an augmented reality display
US9285871B2 (en) 2011-09-30 2016-03-15 Microsoft Technology Licensing, Llc Personal audio/visual system for providing an adaptable augmented reality environment
US8856061B2 (en) * 2011-10-27 2014-10-07 International Business Machines Corporation User experience adjustment in controllable events
US10184798B2 (en) 2011-10-28 2019-01-22 Microsoft Technology Licensing, Llc Multi-stage dead reckoning for crowd sourcing
US11005720B2 (en) * 2011-11-16 2021-05-11 Autoconnect Holdings Llc System and method for a vehicle zone-determined reconfigurable display
US8635637B2 (en) 2011-12-02 2014-01-21 Microsoft Corporation User interface presenting an animated avatar performing a media reaction
US9100685B2 (en) 2011-12-09 2015-08-04 Microsoft Technology Licensing, Llc Determining audience state or interest using passive sensor data
US9429657B2 (en) 2011-12-14 2016-08-30 Microsoft Technology Licensing, Llc Power efficient activation of a device movement sensor module
US8775337B2 (en) 2011-12-19 2014-07-08 Microsoft Corporation Virtual sensor development
US9756571B2 (en) 2012-02-28 2017-09-05 Microsoft Technology Licensing, Llc Energy efficient maximization of network connectivity
US9077813B2 (en) * 2012-02-29 2015-07-07 International Business Machines Corporation Masking mobile message content
US9576593B2 (en) 2012-03-15 2017-02-21 Regents Of The University Of Minnesota Automated verbal fluency assessment
US8898687B2 (en) 2012-04-04 2014-11-25 Microsoft Corporation Controlling a media program based on a media reaction
US9438642B2 (en) 2012-05-01 2016-09-06 Google Technology Holdings LLC Methods for coordinating communications between a plurality of communication devices of a user
CA2775700C (en) 2012-05-04 2013-07-23 Microsoft Corporation Determining a future portion of a currently presented media program
US9591346B2 (en) * 2012-05-14 2017-03-07 Samsung Electronics Co., Ltd. Content delivery system with content sharing mechanism and method of operation thereof
US20140032358A1 (en) * 2012-07-25 2014-01-30 Aro, Inc. Sharing Recommendation Agents
US9817125B2 (en) 2012-09-07 2017-11-14 Microsoft Technology Licensing, Llc Estimating and predicting structures proximate to a mobile device
US9560108B2 (en) 2012-09-13 2017-01-31 Google Technology Holdings LLC Providing a mobile access point
US9183398B2 (en) * 2012-09-20 2015-11-10 Qualcomm Incorporated Content-driven screen polarization with application sessions
US9746990B2 (en) * 2012-09-28 2017-08-29 Intel Corporation Selectively augmenting communications transmitted by a communication device
US10149077B1 (en) * 2012-10-04 2018-12-04 Amazon Technologies, Inc. Audio themes
US9152227B2 (en) 2012-10-10 2015-10-06 At&T Intellectual Property I, Lp Method and apparatus for controlling presentation of media content
US20140143328A1 (en) * 2012-11-20 2014-05-22 Motorola Solutions, Inc. Systems and methods for context triggered updates between mobile devices
GB2508173A (en) * 2012-11-22 2014-05-28 Barclays Bank Plc Identity verification systems and methods
KR102062763B1 (en) * 2012-12-07 2020-01-07 삼성전자주식회사 Method and system for providing information based on context, and computer readable recording medium thereof
US20140195941A1 (en) * 2013-01-10 2014-07-10 International Business Machines Corporation Managing display on a computing system
US20140282786A1 (en) 2013-03-12 2014-09-18 Time Warner Cable Enterprises Llc Methods and apparatus for providing and uploading content to personalized network storage
US9442570B2 (en) 2013-03-13 2016-09-13 Google Technology Holdings LLC Method and system for gesture recognition
US9418354B2 (en) 2013-03-27 2016-08-16 International Business Machines Corporation Facilitating user incident reports
US20140298195A1 (en) * 2013-04-01 2014-10-02 Harman International Industries, Incorporated Presence-aware information system
US9507481B2 (en) * 2013-04-17 2016-11-29 Nokia Technologies Oy Method and apparatus for determining an invocation input based on cognitive load
US10168766B2 (en) 2013-04-17 2019-01-01 Nokia Technologies Oy Method and apparatus for a textural representation of a guidance
US10027606B2 (en) * 2013-04-17 2018-07-17 Nokia Technologies Oy Method and apparatus for determining a notification representation indicative of a cognitive load
US10359835B2 (en) 2013-04-17 2019-07-23 Nokia Technologies Oy Method and apparatus for causing display of notification content
KR102108066B1 (en) * 2013-09-02 2020-05-08 엘지전자 주식회사 Head mounted display device and method for controlling the same
US20150081663A1 (en) * 2013-09-18 2015-03-19 First Principles, Inc. System and method for active search environment
US20150332166A1 (en) * 2013-09-20 2015-11-19 Intel Corporation Machine learning-based user behavior characterization
US20150149390A1 (en) * 2013-11-25 2015-05-28 Palo Alto Research Center Incorporated Method and system for creating an intelligent digital self representation
US9946893B2 (en) 2013-11-26 2018-04-17 Nokia Technologies Oy Method and apparatus for providing privacy profile adaptation based on physiological state change
US20150162000A1 (en) * 2013-12-10 2015-06-11 Harman International Industries, Incorporated Context aware, proactive digital assistant
US9629774B2 (en) 2014-01-14 2017-04-25 Toyota Motor Engineering & Manufacturing North America, Inc. Smart necklace with stereo vision and onboard processing
US10024679B2 (en) * 2014-01-14 2018-07-17 Toyota Motor Engineering & Manufacturing North America, Inc. Smart necklace with stereo vision and onboard processing
US9578307B2 (en) 2014-01-14 2017-02-21 Toyota Motor Engineering & Manufacturing North America, Inc. Smart necklace with stereo vision and onboard processing
US10360907B2 (en) 2014-01-14 2019-07-23 Toyota Motor Engineering & Manufacturing North America, Inc. Smart necklace with stereo vision and onboard processing
US10248856B2 (en) 2014-01-14 2019-04-02 Toyota Motor Engineering & Manufacturing North America, Inc. Smart necklace with stereo vision and onboard processing
US9915545B2 (en) * 2014-01-14 2018-03-13 Toyota Motor Engineering & Manufacturing North America, Inc. Smart necklace with stereo vision and onboard processing
US8782122B1 (en) * 2014-01-17 2014-07-15 Maximilian A. Chang Automated collaboration for peer-to-peer electronic devices
US8782121B1 (en) * 2014-01-17 2014-07-15 Maximilian A. Chang Peer-to-peer electronic device handling of social network activity
US9135849B2 (en) 2014-01-31 2015-09-15 International Business Machines Corporation Variable operating mode HMD application management based upon crowd determined distraction
US9854439B2 (en) 2014-02-07 2017-12-26 First Principles, Inc. Device and method for authenticating a user of a voice user interface and selectively managing incoming communications
US9311639B2 (en) 2014-02-11 2016-04-12 Digimarc Corporation Methods, apparatus and arrangements for device to device communication
US9009516B1 (en) 2014-02-19 2015-04-14 Google Inc. Adjusting a power mode of a wearable computing device based on motion data
US10613642B2 (en) 2014-03-12 2020-04-07 Microsoft Technology Licensing, Llc Gesture parameter tuning
JP6376807B2 (en) * 2014-04-02 2018-08-22 キヤノン株式会社 Display device, display control method, and program
KR20160006417A (en) * 2014-07-09 2016-01-19 엘지전자 주식회사 Mobile terminal and method for controlling the same
WO2016018057A1 (en) 2014-07-31 2016-02-04 Samsung Electronics Co., Ltd. Method and device for providing function of mobile terminal
US10024667B2 (en) 2014-08-01 2018-07-17 Toyota Motor Engineering & Manufacturing North America, Inc. Wearable earpiece for providing social and environmental awareness
CN106575150B (en) 2014-08-16 2020-03-03 谷歌有限责任公司 Method for recognizing gestures using motion data and wearable computing device
US10660039B1 (en) 2014-09-02 2020-05-19 Google Llc Adaptive output of indications of notification data
US9942627B2 (en) * 2014-09-12 2018-04-10 Intel Corporation Dynamic information presentation based on user activity context
US10024678B2 (en) * 2014-09-17 2018-07-17 Toyota Motor Engineering & Manufacturing North America, Inc. Wearable clip for providing social and environmental awareness
US9922236B2 (en) 2014-09-17 2018-03-20 Toyota Motor Engineering & Manufacturing North America, Inc. Wearable eyeglasses for providing social and environmental awareness
USD768024S1 (en) 2014-09-22 2016-10-04 Toyota Motor Engineering & Manufacturing North America, Inc. Necklace with a built in guidance device
US20160162842A1 (en) * 2014-12-04 2016-06-09 Dogpatch Technology, Inc. Messaging system and method
KR102247518B1 (en) 2014-12-23 2021-05-03 삼성전자주식회사 Wearable apparatus, management server, management system having the same and method for controlling thereof
US9576460B2 (en) 2015-01-21 2017-02-21 Toyota Motor Engineering & Manufacturing North America, Inc. Wearable smart device for hazard detection and warning based on image and audio data
CN104635927A (en) * 2015-01-27 2015-05-20 深圳富泰宏精密工业有限公司 Interactive display system and method
US10490102B2 (en) 2015-02-10 2019-11-26 Toyota Motor Engineering & Manufacturing North America, Inc. System and method for braille assistance
US9586318B2 (en) 2015-02-27 2017-03-07 Toyota Motor Engineering & Manufacturing North America, Inc. Modular robot with smart device
US9677901B2 (en) 2015-03-10 2017-06-13 Toyota Motor Engineering & Manufacturing North America, Inc. System and method for providing navigation instructions at optimal times
US9811752B2 (en) 2015-03-10 2017-11-07 Toyota Motor Engineering & Manufacturing North America, Inc. Wearable smart device and method for redundant object identification
US9972216B2 (en) 2015-03-20 2018-05-15 Toyota Motor Engineering & Manufacturing North America, Inc. System and method for storing and playback of information for blind users
US10990888B2 (en) 2015-03-30 2021-04-27 International Business Machines Corporation Cognitive monitoring
US9804679B2 (en) 2015-07-03 2017-10-31 Google Inc. Touchless user interface navigation using gestures
EP3322328A4 (en) * 2015-07-15 2019-03-06 Otorize Ltd. System and method for cognition-dependent access control
US9898039B2 (en) 2015-08-03 2018-02-20 Toyota Motor Engineering & Manufacturing North America, Inc. Modular smart necklace
WO2017025485A1 (en) * 2015-08-07 2017-02-16 Sensomotoric Instruments Gesellschaft Für Innovative Sensorik Gmbh Communication system and method for providing a bionic virtual meeting room
CN105652442A (en) * 2015-12-31 2016-06-08 北京小鸟看看科技有限公司 Head-mounted display equipment and interaction method for head-mounted display equipment and intelligent terminal
US9788200B2 (en) * 2016-02-29 2017-10-10 Motorola Solutions, Inc. Mobile communications device with a private zone and a non-private zone and methods of displaying communications in the same
US10024680B2 (en) 2016-03-11 2018-07-17 Toyota Motor Engineering & Manufacturing North America, Inc. Step based guidance system
US10048929B2 (en) 2016-03-24 2018-08-14 Lenovo (Singapore) Pte. Ltd. Adjusting volume settings based on proximity and activity data
US11025741B2 (en) * 2016-05-25 2021-06-01 International Business Machines Corporation Dynamic cognitive user interface
US9958275B2 (en) 2016-05-31 2018-05-01 Toyota Motor Engineering & Manufacturing North America, Inc. System and method for wearable smart device communications
JP6830206B2 (en) * 2016-06-13 2021-02-17 パナソニックIpマネジメント株式会社 Device control system, information processing device and device control method
US10561519B2 (en) 2016-07-20 2020-02-18 Toyota Motor Engineering & Manufacturing North America, Inc. Wearable computing device having a curved back to reduce pressure on vertebrae
US10432851B2 (en) 2016-10-28 2019-10-01 Toyota Motor Engineering & Manufacturing North America, Inc. Wearable computing device for detecting photography
US10012505B2 (en) 2016-11-11 2018-07-03 Toyota Motor Engineering & Manufacturing North America, Inc. Wearable system for providing walking directions
US10521669B2 (en) 2016-11-14 2019-12-31 Toyota Motor Engineering & Manufacturing North America, Inc. System and method for providing guidance or feedback to a user
US10172760B2 (en) 2017-01-19 2019-01-08 Jennifer Hendrix Responsive route guidance and identification system
US20180260448A1 (en) * 2017-03-10 2018-09-13 Microsoft Technology Licensing, Llc Electronic entity characteristics mirroring
US10735367B2 (en) * 2017-08-03 2020-08-04 Fujitsu Limited Electronic message management based on cognitive load
US11017115B1 (en) * 2017-10-30 2021-05-25 Wells Fargo Bank, N.A. Privacy controls for virtual assistants
US10719832B1 (en) 2018-01-12 2020-07-21 Wells Fargo Bank, N.A. Fraud prevention tool
KR102455056B1 (en) 2018-02-01 2022-10-17 삼성전자주식회사 Electronic device and method for controlling the electronic device thereof
DE102018001322A1 (en) * 2018-02-20 2019-08-22 Fosanis Gmbh Method for generating an interactive message sequence for therapy accompaniment, system
US11036742B2 (en) 2018-03-16 2021-06-15 Motorola Solutions, Inc. Query result allocation based on cognitive load
US10783261B1 (en) * 2018-05-16 2020-09-22 NortonLifeLock Inc. Systems and methods for preventing sensitive information exposure based on a surrounding audience
JP2021529382A (en) 2018-06-19 2021-10-28 エリプシス・ヘルス・インコーポレイテッド Systems and methods for mental health assessment
US20190385711A1 (en) 2018-06-19 2019-12-19 Ellipsis Health, Inc. Systems and methods for mental health assessment
US20190394511A1 (en) * 2018-06-21 2019-12-26 Wells Fargo Bank, N.A. Systems and methods for generating and presenting content-based summaries and recommendations
US11181982B2 (en) 2018-08-01 2021-11-23 International Business Machines Corporation Repetitive stress and compulsive anxiety prevention system
US11328086B2 (en) * 2018-09-06 2022-05-10 Microsoft Technology Licensing, Llc Privacy disclosure
KR102356623B1 (en) * 2019-02-01 2022-01-28 삼성전자주식회사 Virtual assistant electronic device and control method thereof
CN111796680B (en) * 2020-06-29 2023-09-22 青岛歌尔智能传感器有限公司 Control method and device of wearable device, wearable device and medium
US20220174463A1 (en) * 2020-12-02 2022-06-02 West Affum Holdings Corp. Detailed alarm messages and support
US20220375315A1 (en) * 2021-05-18 2022-11-24 Google Llc Adapting notifications based on user activity and environment

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO1990008361A1 (en) * 1989-01-13 1990-07-26 The Scott Fetzer Company Apparatus and method for controlling and monitoring the exercise session for remotely located patients
US5201034A (en) * 1988-09-30 1993-04-06 Hitachi Ltd. Interactive intelligent interface
JPH05260188A (en) * 1992-03-13 1993-10-08 Matsushita Electric Ind Co Ltd Telephone equipment with incoming control function
WO1997003434A1 (en) * 1995-07-12 1997-01-30 Virtual I/O, Inc. Ultra portable computer
JPH0991112A (en) * 1995-07-19 1997-04-04 Toshiba Corp Multimodal interactive device and interactive method
EP0823813A2 (en) * 1994-03-23 1998-02-11 Kopin Corporation Portable communication device

Family Cites Families (45)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5227614A (en) 1986-08-15 1993-07-13 Norand Corporation Core computer processor module, and peripheral shell module assembled to form a pocket size data capture unit
US4916441A (en) 1988-09-19 1990-04-10 Clinicom Incorporated Portable handheld terminal
US4905163A (en) * 1988-10-03 1990-02-27 Minnesota Mining & Manufacturing Company Intelligent optical navigator dynamic information presentation and navigation system
US5349678A (en) 1991-08-21 1994-09-20 Norand Corporation Versatile RF data capture system
US5278946A (en) * 1989-12-04 1994-01-11 Hitachi, Ltd. Method of presenting multimedia data in a desired form by comparing and replacing a user template model with analogous portions of a system
US5032083A (en) 1989-12-08 1991-07-16 Augmentech, Inc. Computerized vocational task guidance system
US5214757A (en) 1990-08-07 1993-05-25 Georesearch, Inc. Interactive automated mapping system
US5781913A (en) 1991-07-18 1998-07-14 Felsenstein; Lee Wearable hypermedium system
US5208449A (en) 1991-09-09 1993-05-04 Psc, Inc. Portable transaction terminal
WO1993014454A1 (en) * 1992-01-10 1993-07-22 Foster-Miller, Inc. A sensory integrated data interface
US5388198A (en) * 1992-04-16 1995-02-07 Symantec Corporation Proactive presentation of automating features to a computer user
US5491651A (en) 1992-05-15 1996-02-13 Key, Idea Development Flexible wearable computer
US5601435A (en) * 1994-11-04 1997-02-11 Intercare Method and apparatus for interactively monitoring a physiological condition and for interactively providing health related information
US5335276A (en) 1992-12-16 1994-08-02 Texas Instruments Incorporated Communication system and methods for enhanced information transfer
GB9315448D0 (en) * 1993-07-26 1993-09-08 Rank Xerox Ltd Recording and retrieval of information relevant to the activities of a user
US5416730A (en) 1993-11-19 1995-05-16 Appcon Technologies, Inc. Arm mounted computer
US5812865A (en) 1993-12-03 1998-09-22 Xerox Corporation Specifying and establishing communication data paths between particular media devices in multiple media device computing systems based on context of a user or users
US5493692A (en) 1993-12-03 1996-02-20 Xerox Corporation Selective delivery of electronic messages in a multiple computer system based on context and environment of a user
US5555376A (en) 1993-12-03 1996-09-10 Xerox Corporation Method for granting a user request having locational and contextual attributes consistent with user policies for devices having locational attributes consistent with the user request
US5566069A (en) 1994-03-07 1996-10-15 Monsanto Company Computer network for collecting and analyzing agronomic data
US5470233A (en) 1994-03-17 1995-11-28 Arkenstone, Inc. System and method for tracking a pedestrian
US5646629A (en) 1994-05-16 1997-07-08 Trimble Navigation Limited Memory cartridge for a handheld electronic video game
US5559520A (en) 1994-09-26 1996-09-24 Lucent Technologies Inc. Wireless information system for acquiring location related information
US6571279B1 (en) * 1997-12-05 2003-05-27 Pinpoint Incorporated Location enhanced information delivery system
EP0718784B1 (en) * 1994-12-20 2003-08-27 Sun Microsystems, Inc. Method and system for the retrieval of personalized information
US5959611A (en) 1995-03-06 1999-09-28 Carnegie Mellon University Portable computer system with ergonomic input device
US5642303A (en) 1995-05-05 1997-06-24 Apple Computer, Inc. Time and location based computing
US5873070A (en) 1995-06-07 1999-02-16 Norand Corporation Data collection system
US6127990A (en) * 1995-11-28 2000-10-03 Vega Vista, Inc. Wearable display and methods for controlling same
US5726660A (en) 1995-12-01 1998-03-10 Purdy; Peter K. Personal data collection and reporting system
JP3216514B2 (en) 1996-01-31 2001-10-09 株式会社デンソー Information service equipment
US5910799A (en) 1996-04-09 1999-06-08 International Business Machines Corporation Location motion sensitive user interface
US5790974A (en) 1996-04-29 1998-08-04 Sun Microsystems, Inc. Portable calendaring device having perceptual agent managing calendar entries
US6047301A (en) 1996-05-24 2000-04-04 International Business Machines Corporation Wearable computer
US6014638A (en) * 1996-05-29 2000-01-11 America Online, Inc. System for customizing computer displays in accordance with user preferences
US5945988A (en) * 1996-06-06 1999-08-31 Intel Corporation Method and apparatus for automatically determining and dynamically updating user preferences in an entertainment system
US5966533A (en) * 1996-06-11 1999-10-12 Excite, Inc. Method and system for dynamically synthesizing a computer program by differentially resolving atoms based on user context data
US5719743A (en) 1996-08-15 1998-02-17 Xybernaut Corporation Torso worn computer which can stand alone
US5938721A (en) 1996-10-24 1999-08-17 Trimble Navigation Limited Position based personal digital assistant
US5902347A (en) 1996-11-19 1999-05-11 American Navigation Systems, Inc. Hand-held GPS-mapping device
US5798733A (en) 1997-01-21 1998-08-25 Northrop Grumman Corporation Interactive position guidance apparatus and method for guiding a user to reach a predetermined target position
US5991687A (en) 1997-07-02 1999-11-23 Case Corporation System and method for communicating information related to a geographical area
US6195622B1 (en) * 1998-01-15 2001-02-27 Microsoft Corporation Methods and apparatus for building attribute transition probability models for use in pre-fetching resources
US6405159B2 (en) * 1998-06-03 2002-06-11 Sbc Technology Resources, Inc. Method for categorizing, describing and modeling types of system users
US6230111B1 (en) * 1998-08-06 2001-05-08 Yamaha Hatsudoki Kabushiki Kaisha Control system for controlling object using pseudo-emotions and pseudo-personality generated in the object

Patent Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5201034A (en) * 1988-09-30 1993-04-06 Hitachi Ltd. Interactive intelligent interface
WO1990008361A1 (en) * 1989-01-13 1990-07-26 The Scott Fetzer Company Apparatus and method for controlling and monitoring the exercise session for remotely located patients
JPH05260188A (en) * 1992-03-13 1993-10-08 Matsushita Electric Ind Co Ltd Telephone equipment with incoming control function
EP0823813A2 (en) * 1994-03-23 1998-02-11 Kopin Corporation Portable communication device
WO1997003434A1 (en) * 1995-07-12 1997-01-30 Virtual I/O, Inc. Ultra portable computer
JPH0991112A (en) * 1995-07-19 1997-04-04 Toshiba Corp Multimodal interactive device and interactive method
US5878274A (en) * 1995-07-19 1999-03-02 Kabushiki Kaisha Toshiba Intelligent multi modal communications apparatus utilizing predetermined rules to choose optimal combinations of input and output formats

Non-Patent Citations (4)

* Cited by examiner, † Cited by third party
Title
LUNT T F ET AL: "KNOWLEDGE-BASED INTRUSION DETECTION", PROCEEDINGS OF THE ANNUAL ARTIFICIAL INTELLIGENCE SYSTEMS IN GOVERNMENT CONFERENCE,US,WASHINGTON, IEEE COMP. SOC. PRESS, vol. CONF. 4, 1989, pages 102 - 107, XP000040018 *
PATENT ABSTRACTS OF JAPAN vol. 018, no. 026 (E - 1491) 14 January 1994 (1994-01-14) *
PATENT ABSTRACTS OF JAPAN vol. 1997, no. 08 29 August 1997 (1997-08-29) *
SATO J ET AL: "Autonomous behavior control of virtual actors based on the AIR model", PROCEEDINGS COMPUTER ANIMATION,XX,XX, 5 June 1997 (1997-06-05), pages 113 - 118-118, XP002122980 *

Cited By (26)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9906474B2 (en) 1998-12-18 2018-02-27 Microsoft Technology Licensing, Llc Automated selection of appropriate information based on a computer user's context
US9559917B2 (en) 1998-12-18 2017-01-31 Microsoft Technology Licensing, Llc Supplying notifications related to supply and consumption of user context data
US9372555B2 (en) 1998-12-18 2016-06-21 Microsoft Technology Licensing, Llc Managing interactions between computer users' context models
US9183306B2 (en) 1998-12-18 2015-11-10 Microsoft Technology Licensing, Llc Automated selection of appropriate information based on a computer user's context
US9443037B2 (en) 1999-12-15 2016-09-13 Microsoft Technology Licensing, Llc Storing and recalling information to augment human memories
US7243130B2 (en) 2000-03-16 2007-07-10 Microsoft Corporation Notification platform architecture
EP1182541A3 (en) * 2000-08-22 2005-11-30 Siemens Aktiengesellschaft System and method for combined use of different display/apparatus types with system controlled context dependant information representation
EP1388093A4 (en) * 2001-05-15 2007-01-24 Nokia Corp Method and business process to maintain privacy in distributed recommendation systems
EP1388093A1 (en) * 2001-05-15 2004-02-11 Nokia Corporation Method and business process to maintain privacy in distributed recommendation systems
WO2003100585A2 (en) * 2002-05-24 2003-12-04 Koninklijke Philips Electronics N.V. Context-aware portable device
WO2003100585A3 (en) * 2002-05-24 2004-07-01 Koninkl Philips Electronics Nv Context-aware portable device
EP1452988A1 (en) * 2003-02-28 2004-09-01 Microsoft Corporation Bounded-deferral policies for guiding the timing of alerting, interaction and communications using local sensory information
EP1606709A1 (en) * 2003-03-26 2005-12-21 Microsoft Corporation Extensible user context system for delivery of notifications
KR100939428B1 (en) * 2003-03-26 2010-01-28 마이크로소프트 코포레이션 Extensible user context system for delivery of notifications
EP1606709A4 (en) * 2003-03-26 2007-12-26 Microsoft Corp Extensible user context system for delivery of notifications
US8413167B2 (en) * 2007-10-01 2013-04-02 Microsoft Corporation Notifying a user of access to information by an application
US10277479B2 (en) 2010-05-11 2019-04-30 Nokia Technologies Oy Method and apparatus for determining user context
US9497309B2 (en) 2011-02-21 2016-11-15 Google Technology Holdings LLC Wireless devices and methods of operating wireless devices based on the presence of another person
US9372997B2 (en) 2013-12-23 2016-06-21 Google Inc. Displaying private information on personal devices
CN105940411A (en) * 2013-12-23 2016-09-14 谷歌公司 Displaying private information on personal devices
WO2015099912A1 (en) * 2013-12-23 2015-07-02 Google Inc. Displaying private information on personal devices
EP3340102A1 (en) * 2013-12-23 2018-06-27 Google LLC Displaying private information on personal devices
US9832187B2 (en) 2014-01-07 2017-11-28 Google Llc Managing display of private information
CN106575344A (en) * 2014-08-20 2017-04-19 谷歌公司 Displaying private information on personal devices
WO2016028538A1 (en) * 2014-08-20 2016-02-25 Google Inc. Displaying private information on personal devices
US10311249B2 (en) 2017-03-31 2019-06-04 Google Llc Selectively obscuring private information based on contextual information

Also Published As

Publication number Publication date
US7512889B2 (en) 2009-03-31
US20060053377A1 (en) 2006-03-09
US6466232B1 (en) 2002-10-15
AU2190200A (en) 2000-07-03
DE69902487D1 (en) 2002-09-12
EP1276037A3 (en) 2009-04-15
DE69902487T2 (en) 2002-12-19
US6874127B2 (en) 2005-03-29
EP1151372A1 (en) 2001-11-07
US20020078204A1 (en) 2002-06-20
EP1151372B1 (en) 2002-08-07
EP1276037A2 (en) 2003-01-15

Similar Documents

Publication Publication Date Title
US6874127B2 (en) Method and system for controlling presentation of information to a user based on the user's condition
US6549915B2 (en) Storing and recalling information to augment human memories
US7614001B2 (en) Thematic response to a computer user's context, such as by a wearable personal computer
US7055101B2 (en) Thematic response to a computer user's context, such as by a wearable personal computer
US7080322B2 (en) Thematic response to a computer user's context, such as by a wearable personal computer
US7076737B2 (en) Thematic response to a computer user's context, such as by a wearable personal computer
US6842877B2 (en) Contextual responses based on automated learning techniques
US11218848B2 (en) Messaging enhancement with location information
US7403972B1 (en) Method and system for enhanced messaging
US20110126119A1 (en) Contextual presentation of information
US20200322301A1 (en) Message delivery and presentation methods, systems and devices using receptivity
CN110753514A (en) Sleep monitoring based on implicit acquisition for computer interaction
EP1314102B1 (en) Thematic response to a computer user's context, such as by a wearable personal computer
US20210019108A1 (en) Multi-application control of augmented reality audio

Legal Events

Date Code Title Description
AK Designated states

Kind code of ref document: A1

Designated state(s): AE AL AM AT AU AZ BA BB BG BR BY CA CH CN CU CZ DE DK EE ES FI GB GE GH GM HR HU ID IL IN IS JP KE KG KP KR KZ LC LK LR LS LT LU LV MD MG MK MN MW MX NO NZ PL PT RO RU SD SE SG SI SK SL TJ TM TR TT UA UG US UZ VN YU ZA ZW

AL Designated countries for regional patents

Kind code of ref document: A1

Designated state(s): GH GM KE LS MW SD SL SZ TZ UG ZW AM AZ BY KG KZ MD RU TJ TM AT BE CH CY DE DK ES FI FR GB GR IE IT LU MC NL PT SE BF BJ CF CG CI CM GA GN GW ML MR NE SN TD TG

121 Ep: the epo has been informed by wipo that ep was designated in this application
DFPE Request for preliminary examination filed prior to expiration of 19th month from priority date (pct application filed before 20040101)
WWE Wipo information: entry into national phase

Ref document number: 1999966337

Country of ref document: EP

REG Reference to national code

Ref country code: DE

Ref legal event code: 8642

WWP Wipo information: published in national office

Ref document number: 1999966337

Country of ref document: EP

WWG Wipo information: grant in national office

Ref document number: 1999966337

Country of ref document: EP