WO2013024276A1

WO2013024276A1 - Optimised context-awareness on mobile devices

Info

Publication number: WO2013024276A1
Application number: PCT/GB2012/051971
Authority: WO
Inventors: Ian Anderson; Michael Mccarthy
Original assignee: Overlay Media Limited
Priority date: 2011-08-12
Filing date: 2012-08-13
Publication date: 2013-02-21
Also published as: GB201113920D0

Abstract

A context of a mobile device is determined by: when the mobile device is stationary, determining that the mobile device is at a place of interest; when the mobile device is not stationary, determining that the mobile device is travelling; and determining a mode of travel. The step of determining the mode of travel comprises: selecting an activity recognition method from a plurality of available activity recognition methods; and determining the mode of travel using the selected activity recognition method. The available activity recognition methods comprise detecting signals transmitted from wireless beacons (such as base stations of a cellular telephone network), using a satellite positioning system to determine a position of the mobile device, and using at least one accelerometer to detect movement of the mobile device. The step of selecting an activity recognition method comprises: determining the mode of travel using the activity recognition method that comprises detecting signals transmitted from wireless beacons; and when the activity recognition method that comprises detecting signals transmitted from wireless beacons indicates that the mode of travel has changed, selecting another activity recognition method from the available activity recognition methods.

Description

Optimised Context-Awareness on Mobile Devices

Field of the Invention

The present invention relates to methods, apparatus and programs for inferring a state of activity of a carrier of a mobile communication device, and to methods, apparatus and programs for calibrating, and selecting an appropriate method and algorithm for inferring a state of activity of carrier of a mobile communication device.

Background to the Invention

In recent years the term Point of Interest, or POI as it is often abbreviated to, has become commonplace. A POI represents a location that has some relevance or interest to people. From the perspective of a person POIs represent everyday locations, for example, the home, the workplace, shops, and so on. A person visiting a POI would generally describe their location as being at the POI, e.g. "I'm at the bank" or "I'm at the coffee shop". Membership to a POI (being located at a POI) occurs when the person is within the boundaries of the POI, e.g. inside the coffee shop.

The term activity is used to describe a state of motion that a person is undertaking. Walking, running, cycling, driving, travelling on a train, remaining stationary are all examples of activities.

The activity being performed at the POI may vary over time. For example, consider a driver of a motor vehicle stopping at a fuelling station to fill up the fuel of their vehicle. The POI is the fuelling station and whilst the driver is within the boundaries of the fuelling station they can be considered to be at that POI. During their time at this POI it is likely that their current activity would transition through the following stages. Initially they would be standing still whilst the car is filled with fuel. Then they would walk to the service station assistant, remain stationary whilst paying for the fuel before walking back to their car. Therefore at the location of the POI the user will perform the following activities walk, remain stationary and driving (entering and leaving the forecourt). A visit to a coffee shop is likely to involve two activities, walking and queuing to pay for the coffee and sitting (remaining stationary) whilst consuming the coffee. All of these activities occur whilst the user is within the boundaries of the POI and the specific activities typically involve the user only moving at a slow speed or at times stationary.

Given that a person is typically stationary (or moving at a slow speed) whilst located at a POI we can use the state of activity to infer when the user is located at a POI. If the user is stationary they can be considered to be located at a POI.

Given that an activity state can be used to indicate that a person is located at a POI a moving activity state (e.g. travelling in a motor car) can be used to indicate that the user is not currently at a POI. This means that a person will always be in one of two states: located at a POI and not located at a POI. We can treat the second state as representing a transition between POIs. This is based upon the principle that the user is either at a POI or transitioning (travelling) to the next POI.

Given that a person must travel if they are to move between different POIs (transition from one POI to another), the method of travel provides an indication of the likelihood of transitioning to a particular POI. For example, the process of going to work, a transition from the Home (POI) to the Workplace (POI). This journey might regularly be completed by car whereas a visit from Home (POI) to a local shop (POI) may usually be completed by walking. Knowing that the person has just left home and is travelling in their car means that they are unlikely to be visiting the local shop.

Equally, being located in a specific POI may influence the next POI; the POI that the user will travel to next. For example, a person may only ever walk to the Pub; therefore if they are at the office they may drive home (POI) first before walking on to the local Pub.

Using the history of a persons' visits to POIs and transitions between POIs will enable predictions to be made of that persons' future intents. For example, it would be possible to estimate what time a person will get back from work (return to the Home POI) by looking at previous arrival times. In summary, a person is considered at a POI when they are stationary (or moving at a slow speed within the boundary of the POI), if they are not stationary they are considered to be in a transition state (moving between POIs), information about the visit duration (time spent at a POI), time and day of visit can be used to infer the semantic interpretation of a POI, the transition history (movements between POIs that a person has performed) can be used to indicate the POI that a user is next likely to visit, the POI visit history transition history can be used to make predictions about future intents.

In order to implement this behaviour on a mobile device a number of problems need to be solved. These include: how to recognise states of activity, how to create identifiers for POIs that can be used to recognise a return to a POI, how to make semantic interpretation of POIs, how to make predictions of future user intents. These problems need to be solved in consideration of the constraints of the mobile device e.g. a limited amount of battery power that restricts the amount of processing that can be performed.

We now provide a background overview of methods for inferring the current activity of the carrier of a mobile device and methods for determining the location of a mobile device.

Mode of travel when transitioning between POIs can be detected using sensors equipped to a mobile device. We now provide a background overview of methods for inferring the current activity of the carrier of a mobile device and methods for determining the location of a mobile device.

Mode of travel when transitioning between POIs can be detected using sensors equipped to a mobile device. In this disclosure we discuss three methods for inferring the current activity of the carrier of a mobile device.

For further background we now describe the implementation of a system that automatically recognises Places Of Interest (POIs), this system is referred to as a context service. Our invention builds on this to provide an optimised approach to recognising the activities performed whilst undertaking transitions between POIs. Context Service

The context service running on the cellular phone provides the core POI and Transition functionality including: recognising a person returning to previously visited POIs, automatically creating identifiers for new (previously unknown) POIs, identifying transitions between POIs. In addition to this the context service running on the cellular phone will also record (persist to storage on the cellular phone) data relating to POIs, transitions and a person's movements throughout this POI network.

Figure 7 provides an illustration of a collection of POIs and transitions between those POIs. This is referred to as the POI network. As can be seen a POI network can be considered as a directed graph with nodes representing POIs and arcs representing transitions. In Figure 7 1, 2 and 3 represent POIs in the network. The arcs Tl, T2, T3 and T4 represent transitions between these POIs (nodes). The following example illustrates how this representation translates to a real world scenario. POI 1 may represent a persons' home. Transition Tl may represent the person driving to work. POI 2 may represent their office. Transition T2 may indicate them walking to a local restaurant (POI 3) to have some food at lunchtime. Transition T3 may represent the walk from the restaurant back to the office. Transition T4 may represent the drive from the office (POI 2) back to the home (POI 1).

As a person visits previously unknown POIs their POI network is extended and the transitions (arcs) that link the newly discovered POI with the previously departed POI are also added.

The POI network provides an overview of the POI topology. It does not provide a representation of a persons' movement around the network, there is no notion of time, nor any indication of transition frequency or visit duration.

Figure 8 presents a state chart that describes the overall workflow of the context service. In Figure 8, 1 represents the transitioning between POIs state. When the user is detected as being stationary 2 they enter state 3 the stationary state. Whilst stationary 3 the context service samples wireless data to create a candidate identifier 4 that can be matched against existing POI identifiers 5. Once the context service has created the candidate identifier it is compared to existing POI identifiers 5. If there is a match 10 then the user is deemed to be located at a POI 10 and they will remain in the state until they leave the POI 11 , commencing a transition. If there were no matches then the user is deemed to have arrived at a new POI 6 and this new POI needs to be added to the user's POI network 7. Once the POI has been added the user is deemed to be located at the POI 9.

An object of the present invention is to minimise power consumption whilst recognising the activities performed during a transition between two POIs.

Another object of the present invention is to switch between different methods for inferring the activity state of the carrier of the mobile device whilst fulfilling the needs of the applications and services running on the mobile device and whilst also minimising power consumption.

Summary of the Invention

This invention relates to a method for recognising activities that the carrier of a mobile device may be undertaking, e.g. walking, driving and so on. This invention minimises the amount of computation and battery power that is used in the process of inferring the current activity by automatically switching between different activity recognition technologies. This invention also relates to calibration of the activity recognition technologies. A trained GPS system can be used to calibrate an accelerometer system. Equally a trained accelerator method could be used to calibrate CSF. It will be apparent to those skilled in the relevant art that any combination of the above is also possible.

In this disclosure we discuss three methods for inferring the current activity of the carrier of a mobile device.

The first described method involves the use of velocity data obtained from a GPS receiver. This method maps velocity to activities for example, a high velocity - upwards of 25 miles per hour is likely to indicate that the carrier of the GPS receiver is travelling in a motor car. A low velocity such as between two and four miles an hour is likely to indicate that the carrier of the GPS receiver is walking. Patterson et al. (D. Patterson, L. Liao, D. Fox, and H. Kautz. Inferring highlevel behavior from low-level sensors. In UBICOMP 2003: The Fifth International Conference on Ubiquitous Computing., October 2003.) used a GPS receiver to distinguish between different modes of transport such as walking, driving or taking a bus. Data was collected over a three month period and daily patterns of behaviour were learnt using a graph- based Bayes filter. The mode of transportation was then estimated using a particle filter. This work supports a higher level prediction to be made regarding the purpose of a user's journey.

The second method for inferring the current state of the carrier of a mobile device uses accelerometer signal data. Movement (stationary or moving) can be inferred by analysing signal data collected from accelerometers. US 2006/0187847 (Cisco Technology, Inc) discloses a method for detecting the state of a mobile device using accelerometer signals. The Sensay project (D. P. Siewiorek, A. Smailagic, J. Furukawa, A. Krause, N. Moraveji, K. Reiger, J. Shaffer, and F. L. Wong. Sensay: A context-aware mobile phone. In 7th International Symposium on Wearable Computers, ISWC, pages 248-249. IEEE Computer Society, 2003.) used three accelerometers to capture the motion of the user. The accelerometers were fitted inside a sensor box that was taped to the user's abdomen. This was used to distinguish between states of low activity such as sitting, medium activity such as walking and high activity such as running. The MIThril project (The Context Aware Cell Phone Project, MIT. http://www.media.mit.edu/wearables/mithril/phone.html/ March, 2006.) also used an accelerometer to distinguish between similar activities including cycling. Lukowicz et al. (P. Lukowicz, J. A. Ward, H. Junker, M. Stager, G. Troster, A. Atrash, and T. Starner. Recognizing workshop activity using body worn microphones and accelerometers. In Pervasive, pages 18-32, 2004.) combined the use of accelerometers with microphones worn on the body. This allowed them to distinguish between an increased range of activities. This was demonstrated in a carpenter's workshop by tracking the progress of an assembly task.

The third method for recognising the current activity of the carrier of a mobile device is to use patterns of Cell-ID and signal strength fluctuation. In this disclosure we refer to this method as "Cell and Signal Strength Fluctuation (CSF)". This method was first presented by Anderson and Muller (Ian Anderson, Henk Muller, Context Awareness via GSM Signal Strength Fluctuation. In the 4th International Conference on Pervasive Computing, Late breaking results. ISBN 3-85403-207-2, pp. 27-31. May 2006). This method used minimal battery power and proved a reliable method for identifying walking, driving and remaining stationary.

We now provide a brief background of the Global System for Mobile Telecommunications (GSM) and how the CSF method for inferring the current activity of the carrier of a mobile device works.

The Global System for Mobile Telecommunications (GSM) is currently the most popular cellular network standard, with over a billion users worldwide. In the US and Canada, GSM networks typically operate in the 850MHz and 1900MHz bands, whilst elsewhere the 900MHz and 1800MHz bands are typically used. Channels are spaced throughout these bands at 200 KHz intervals creating 124 separate channels. On each of these channels GSM networks operate a Time-Division Multiple Access (TDMA) system enabling multiple mobile stations to share the same channel. A cell is allocated a number of channels depending on the predicted usage for the area that it serves. Usage is estimated based on radio surveys and the expected customer base for that area. Once a usage model has been constructed a network designer selects the number and types of cell required to provide coverage for the area. In dense urban environments a high number of micro (short-range) cells are used, thus increasing network capacity. In contrast, service in rural areas is provisioned by macro cells with a coverage range of up to 35km.

In order to provide support for roaming, a mobile communication device, such as a GSM cell phone, monitors the cell currently serving it and a number, typically six or seven, of neighbouring cells, and maintains a list of the monitored cells. When the mobile communication device is stationary, this list typically varies only minimally, in that only a limited number of individual cells (perhaps seven or eight, for example) appear on the list during a given time period. However, when the mobile communication device is moving, the list of monitored cells changes, particularly in metropolitan environments with a large number of cells, such that a larger number of individual cells (perhaps ten or more, for example) appear on the list during a given time period. When a large geographic area is covered the number of cells monitored increases. The precise number of cells monitored depends on both the type of environment and the speed of the carrier of the mobile communication device. In metropolitan environments, for example, there is typically a high number of micro cells, whereas rural environments with lower populations typically require only a few macro-cells. In a given environment, a specific increase in the number of cells is observed as the speed of movement of the carrier of the mobile communication device increases.

Hence a change in the list of monitored neighbouring cells typically indicates a change in the position of the mobile communication device.

One of the parameters monitored by the mobile communication device is the strength of signals received from base transceiver (transmitter-receiver) stations (BTS) associated with the serving cell and the neighbouring cells. When the mobile communication device is stationary, there is typically only a small amount of variation in the strength of these received signals during a given time period. In contrast, when the mobile communication device is moving, the strength of the received signals is subject to a larger degree of variation during a given time period. Thus, variations in the received signal strength are typically indicative of movement of the mobile communication device. Further detail of the CSF method is described in:

Ian Anderson, Henk L Muller, Practical Context Awareness for GSM Cell Phones. International Symposium on Wearable Computing (ISWC) 2006. ISBN 1-4244-0597-1, pp. 127-128. October 2006,)

Ian Anderson, Henk Muller, Practical Activity Recognition using GSM Data. CSTR-06-016, Department of Computer Science, University of Bristol. July 2006

Ian Anderson, Henk Muller, Context Awareness via GSM Signal Strength Fluctuation, the 4th International Conference on Pervasive Computing, Late breaking results. ISBN 3-85403-207-2, pp. 27-31. May 2006

In the table below (Table 1) the characteristics and behaviours of the three different methods for inferring the current activity of the carrier of a mobile device are presented. Column 1 represents an attribute of activity recognition. Column 2 represents the performance of the GPS approach for each attribute. Column 3 represents the performance of the accelerometer approach for each attribute. Column 4 represents the performance of the CSF approach for each attribute. It will be apparent to those skilled in the relevant art that the classification of the performance levels for each attribute are intended to provide a broad assessment.

GPS Accelerometer CSF

Power High High Low

consumption

Requires Yes. A GPS Yes. An No

additional receiver accelerometer

hardware

Works indoors Outdoors only Yes Yes

and outdoors

Time to detection 15 seconds 1 second 30 seconds of movement assuming a fix

has already been obtained

Time to 30 seconds 10 seconds 60 seconds identifying

activity

Table 1. Broad characteristics of activity recognition methods

The table below (Table 2) presents a list of the activities that can typically be sensed using each of the activity recognitions methods. Column 1 represents the activity to be sensed. Columns 2, 3 and 4 represent the performance of each method i.e. whether the method can sense the activity. It will be apparent to those skilled in the relevant art that it is possible to optimise the activity recognition technologies to infer additional activities.

Table 2. Sensing capabilities

(1) GPS may identify running as cycling depending on the speed that the user is running/cy cling. A slow cycle ride would appear the same as a fast run.

(2) CSF will frequently identify running as cycling and vice versa.

As the description of the three different activity recognition methods and Table 1 and Table 2 show, each method has its' merits and limitations. There is no single perfect method for recognising activities that can be run continuously without depleting the battery charge of a mobile device. This invention addresses this problem by minimising power consumption by automatically swapping between different activity recognition techniques dependent on factors such as: the activity to be sensed, whether the mobile device is indoors or outdoors, the current charge of the battery, the position of the user, the semantic meaning of the position of the user (are they at home).

In this disclose we use the term "controller" to refer to the implementation of this invention that is responsible for automatically swapping between different activity recognition techniques dependent on the above factors.

A particular advantage of the invention described in the above embodiments is that power consumption is minimised in an automatic fashion. Turning off the power to a GPS receiver module when it is not needed results in the more efficient use of stored power. This extends the time duration before the battery needs to be recharged. Expecting the carrier of a mobile device equipped with a GPS receiver to repeatedly change the power settings is not practical. Therefore in order for the controller to know when it can turn off ancillary devices such as the GPS receiver the controller needs to know whether they are going to be used. It is clear there is little point providing power to a GPS receiver equipped to a cellular phone in an indoor environment where there is no clear view of the sky. Equally if the user is stationary then the GPS signals provide no new information. Therefore in order to realise these power saving optimisations the controller needs a low- power method for determining context, e.g. CSF. If the controller were to turn off the GPS receiver and use accelerometer based activity recognition techniques then the amount of power would not be as great as if CSF were used instead of the accelerometer technique. CSF does however lack the reliability of accelerometer based activity recognition. This invention optimises the reliability of activity recognition whilst minimising unnecessary power consumption. Brief Description of the Drawings Various embodiments of the invention will now be described with reference to the attached figures in which:

Figure 1 provides a simplified view of a GSM cellular network; Figure 2a illustrates the total number of identifiers associated with cells monitored by a GSM cell phone for different states of activity;

Figure 2b illustrates the total received signal strength fluctuation of a serving cell and neighbouring cells monitored by a GSM cell phone for different states of activity;

Figure 3 is a flow diagram of a method of operation of a cellular phone according to the first embodiment of the invention; Figure 4 is a flow diagram of a method of operation of a cellular phone according to the second embodiment of the invention;

Figure 5 is a flow diagram of a method of operation of a cellular phone according to the third embodiment of the invention

Figure 6 is a flow diagram of a method of operation of a cellular phone according to the fourth embodiment of the invention

Figure 7 is a conceptual view of the data produced by the context service according to the first embodiment of the invention; Figure 8 is a flow diagram of a method of operation of a context service according to the first embodiment of the invention;

Description of the Embodiments

First Embodiment.

The context service described in the earlier background section works on the assumption that the user is in one of two states: located at a POI or transitioning between POIs. Figure 7 presents a view of a POI network showing POIs (1, 2, 3) and transitions between POIs (Tl, T2, T3, T4). Whilst the user is transitioning between POIs it is useful to understand the activities (walking, driving etc) that the user performs. For example, in Figure 7 if person walks from POI 2 to POI 1 but drives from POI 2 to POI 3 then knowledge that the person is walking can be used to infer that the person is likely to be headed (transitioning to) POI 1. As discussed earlier there is no single perfect activity recognition technique capable of recognising all activities and also certain techniques are, in terms of power consumption, expensive. In this invention focus is placed on recognising activities that are undertaken whilst transitioning between POIs in the most power efficient way possible.

Figure 1 illustrates the general architecture of a Global Systems for Mobile Communication (GSM) network. In his figure the cellular phone 1 communicates with the Base Transceiver Station (BTS) 2 that provides the current serving cell 5a. A plurality of neighbouring cells 5b...5 g provide support for roaming. These neighbouring cells are managed by Base Transceiver Station's 3a...3f. In this embodiment the base transceiver stations 2 and 3a...3f are standard base transceiver stations for the Global System for Mobile Communications (GSM). GSM is well known and will not be described in further detail. In this embodiment the cellular phone 1 also includes a conventional Global Positioning System (GPS) receiver that is able to determine its location using signals obtained from GPS satellites 4a...4d forming part of the GPS system. The cellular phone 1 also contains a 3D accelerometer (not shown in Figure 1).

In this embodiment, the cellular phone 1 monitors the strength of the signals received from the base transceiver stations (BTS) associated with the serving cell 5a and the neighbouring cells 5b...5g. When the cellular phone 1 is stationary, there is typically only a small amount of variation in the strength of these received signals during a given time period. In contrast, when the mobile communication device is moving, the strength of the received signals is subject to a larger degree of variation during a given time period. Thus, variations in the received signal strength are typically indicative of movement of the mobile communication device. This is illustrated in Figure 2. Figure 2 shows a total signal strength fluctuation of the signals received from the serving BTS and the BTSs associated with the identifiers appearing in the list of candidates by the cell phone used in the experiment during a rolling 15 second time window. The method used to calculate the total signal strength fluctuation comprised measuring the received signal strength from the serving BTS and from each of the BTSs appearing in the list of candidates at one second intervals during the time window, calculating a signal strength fluctuation value for each BTS by subtracting the minimum signal strength received during the time window from the maximum signal strength received during the time window, and summing the BTS signal strength fluctuations. This method was performed by an algorithm running on the cell phones used by the volunteers, which is represented by the following pseudo-code. Set a short time interval (SAMPLE PERIOD) (in this example 1 second). Set a longer time interval defining the time window size, i.e. the number of samples to assess together (WINDOW PERIOD) (in this example 15 seconds).

Every SAMPLE PERIOD, perform the following steps:

Record the signal strength levels for all of the BTSs that are currently being monitored (neighbouring and serving);

For each BTS compare the new signal strength level with the minimum and maximum signal strength levels previously recorded for that BTS;

If the new signal strength level is greater than the previous maximum then

Set the maximum signal strength level to the new signal strength level;

If the new signal strength level is less than the previous minimum then

Set the minimum signal strength level to the new signal strength level.

(It will be understood that if a BTS only appears on the list of candidates only once then the signal strength fluctuation for that BTS will be zero).

Every WINDOW PERIOD, perform the following steps: For each BTS, calculate the amount of signal strength fluctuation by subtracting the minimum signal strength level that was observed during the WINDOW PERIOD from the maximum signal strength level that was observed during the same WINDOW PERIOD.

Add these individual levels of signal strength fluctuation to produce a single number representing the overall level of signal strength fluctuation for the given WINDOW PERIOD.

Figure 2a shows the total number of identifiers (i.e. ARFCNs) associated with cells appearing in the list of candidates during a rolling 15 second time window, for the activity states stationary, walking and travelling in a motor car. In the stationary activity state, typically only six or seven identifiers appeared in the list of candidate. Typically, more identifiers appeared in the list of candidates in the walking and travelling in a motor car, states of activity, and both of these states of activity shared similar patterns. It is thought that although a greater distance was covered in the motor car than when walking, the number of identifiers appearing in the candidate list was similar for both of these states of activity because of a hardware limitation of the cell phone used to collect the data and the need to avoid repeated changing of the neighbouring cells (thrashing). Hence moving at a slower speed through a metropolitan environment gives the cell phone more opportunity to detect and assess the suitability of neighbouring cells.

Figure 2b shows that it is relatively easy to distinguish between the walking and stationary states of activity by analysing signal strength fluctuation alone. However, at times, the walking and travelling in a motor car, states of activity share similar signal strength patterns. By comparing the fluctuation with GPS traces from the GPS receivers used by the volunteers, it was found that the drops between high spikes of fluctuation typically occurred while waiting at areas of traffic control or road junctions. Hence the graph of Figure 2b reflects the stop-start nature of driving in metropolitan environments. While travelling in a motor car at constant but low speeds (typically below 30mph), the signal strength fluctuation patterns were found to be similar to those obtained while walking. This reflects the speed of travel, but does not represent the mode of travel.

Thus, data received by a mobile communication device such as a GSM cell phone can be processed and used to infer a state of activity of a person carrying the mobile communication device.

As explained above, it is relatively easy to distinguish between the walking and stationary states of activity, and thus to infer the state of activity of a carrier of a mobile communication device, by analysing signal strength fluctuation or the number of BTS identifiers appearing in the list of candidates. However it is difficult to distinguish between the walking and travelling in a motor car, states of activity by analysing these parameters alone. Machine learning techniques can be used to alleviate this problem, by classifying patterns of signal strength fluctuation or of candidate list variation as occurring while the carrier of the mobile communication device is in specific states of activity. This is discussed in more detail in (Ian Anderson, Henk L Muller, Practical Context Awareness for GSM Cell Phones. International Symposium on Wearable Computing (ISWC) 2006. ISBN 1-4244-0597-1, pp. 127-128. October 2006, Ian Anderson, Henk Muller, Practical Activity Recognition using GSM Data. CSTR-06-016, Department of Computer Science, University of Bristol. July 2006, Ian Anderson, Henk Muller, Context Awareness via GSM Signal Strength Fluctuation, the 4th International Conference on Pervasive Computing, Late breaking results. ISBN 3-85403-207-2, pp. 27-31. May 2006) and hence will not be discussed in more detail here. We refer to the process of determining activity using the above method as Cell and Signal Strength fluctuation (CSF).

We refer to the process of determining activity using signal data collected from accelerometer as the "accelerometer based activity recognition".

We refer to the process of determining activity using GPS velocity data as "GPS based activity recognition". We use the term "seeding" to refer to the process of confirming the current activity being undertaken by the carrier of the cellular phone 1 to the controller. Although not strictly necessary, seeding is a useful method for optimising the use activity sensing technologies. CSF suffers from reliability issues when distinguishing between walking and driving. However an accelerometer based approach is far more reliable at distinguishing between these two activities. It makes sense to use the accelerometer based technique to confirm the activity and then switch to CSF to minimise power consumption.

The embodiment illustrated in Figure 3 fuses CSF, accelerometer and GPS activity recognition techniques that minimises power consumption whilst providing activity recognition sensing capabilities. Figure 3 represents a method of operation of the cellular phone 1 as illustrated in Figure 1. After the controller of the cellular phone runs CSF SI and determines that the cellular phone 1 has changed activity state S2, e.g. the cellular phone is no longer stationary. After determining a state change the controller checks at S3 if the GPS receiver is switched on. If the controller ascertains that the GPS receiver is switched on the controller then checks at S4 if the GPS receiver has a positional fix. (The process of decoding the GPS signals is well known and will therefore not be described here). If the controller ascertains that the GPS receiver has a positional fix it then runs the GPS based activity recognition method for determining the current activity S5. After determining the current activity using the GPS based activity recognition method S5. The controller then ascertains of the current activity is travelling in a motor car S7. If the current activity is that of travelling in a motor car then the controller "seeds" the CSF mode as driving Sl l and stops all activity recognition methods apart from CSF S12. If however the controller ascertains that the current activity is walking (S8) then the controller "seeds" the CSF mode as walking S10 and stops all activity recognition methods apart from CSF S12.

If the controller determines that either the GPS receiver is switched off S3 or that the GPS receiver is switched on but it does not currently have a positional fix S4. Then the controller runs the accelerometer based activity recognition method S6. The controller then ascertains of the current activity is travelling in a motor car S7. If the current activity is that of travelling in a motor car then the controller "seeds" the CSF mode as driving Sl l and stops all activity recognition methods apart from CSF S12. If however the controller ascertains that the current activity is walking (S8) then the controller "seeds" the CSF mode as walking S10 and stops all activity recognition methods apart from CSF S12. If the controller determines that the current activity is not driving S7 and is not walking S8 then the controller "seeds" the CSF mode as remaining stationary S9 and stops all activity recognition methods apart from CSF S12.

Second Embodiment

A second embodiment will now be described in which the minimisation of power consumption and activity recognition optimisation method described in the first embodiment is modified to take account of calibrating CSF against GPS/accelerometer based activity recognition methods. The remaining features of the second embodiment are the same as for the first embodiment. Figure 4 represents the method of operation of cellular phone 5 of the second embodiment.

In order to calibrate CSF so that it is suitable for use in a given environment it is necessary to learn the levels of signal strength fluctuation and candidate list variation that are generated whilst undertaking activities such as walking and travelling in a motor car. In order to do this measurement samples of signal strength fluctuation and candidate list variation (changes to the list of monitored cells - current and neighbouring) must be collected whilst undertaking the activities to be sensed. The combination of signal strength fluctuation and candidate list variation that is most likely to occur whilst undertaking those activities can then be learned.

In order to do this it is necessary to be able to identify when the carrier of the mobile communication device is undertaking the activities that it is desired to recognise. To do this the carrier can be asked to prompt when they are performing specific activities. For example, as the carrier leaves their house to drive to work the "travelling in a motor car" option on a training menu of the mobile communication device can be selected. By doing this the user is indicates to the activity recognition algorithm that they are now travelling in a motor car and the current cellular data can be used to learn the patterns of signal strength fluctuation and candidate list variation for travelling in a motor car. This approach is undesirable as it requires intervention from the user. In this embodiment alternative activity sensing methods are used to provide automatic CSF calibration.

This figure (Figure 4) is almost identical to Figure 3. The only differences between the two figures relate to S13 and S14. Upon using GPS/accelerometer based activity recognition methods to determine the current activity the controller will "seed" CSF S9, S10 and SI 1. CSF will be then run alongside either GPS based activity recognition or accelerometer based activity recognition. During this time the controller will collect calibration data S13. This duration of the process of collecting calibration data is controlled by the controller. Two minutes is a sufficient time period to learn the parameters needed for CSF. This time period could however be extended by the controller. Once sufficient calibration data has been collected the controller stops all activity recognition methods apart from CSF S14.

This approach minimises power consumption by using the activity recognition method that requires the least amount of power (CSF) whilst providing high activity recognition reliability and providing support for CSF to be optimised for use in specific environments.

Third Embodiment

In an alternative embodiment the cellular phone 1 does not contain a GPS receiver. The method of operation is shown in Figure 5.

After determining (S2) that the state has changed, the controller determines using CSF measurements whether the cellular phone is moving (SI 5). If not, the process ends (SI 6), but, if it is, the process passes to S6, in which the accelerometer method is run.

Apart from the use of the GPS based activity recognition method this method of operation is identical to the first embodiment.

Fourth Embodiment

In an embodiment the controller on the cellular phone 1 does not use "seeding". The method of operation is shown in Figure 6.

After the controller of the cellular phone runs CSF SI and determines that the cellular phone 1 has changed activity state S2, e.g. the cellular phone is no longer stationary. After determining a state change the controller checks at S3 if the GPS receiver is switched on. If the controller ascertains that the GPS receiver is switched on the controller then checks at S4 if the GPS receiver has a positional fix. (The process of decoding the GPS signals is well known and will therefore not be described here). If the controller ascertains that the GPS receiver has a positional fix it then runs the GPS based activity recognition method for determining the current activity S5. After determining the current activity using the GPS based activity recognition method S5. The controller then ascertains of the current activity is travelling in a motor car S7. If the current activity is that of travelling in a motor car then the controller remains in this state until the GPS based activity recognition method detects that the carrier of the cellular phone 1 is no longer travelling in a motor car. At this point the controller will then ascertain whether the carrier of the cellular phone 1 is walking S8. If the controller ascertains that the carrier of the cellular phone 1 is walking then the controller remains in this state until the GPS based activity recognition method detects that the carrier of the cellular phone 1 is no longer walking. At this point the controller stops all activity recognition methods S19 and runs CSF SI .

If the controller determines that either the GPS receiver is switched off S3 or that the GPS receiver is switched on but it does not currently have a positional fix S4. Then the controller runs the accelerometer based activity recognition method S6. The controller then ascertains of the current activity is travelling in a motor car S7. If the current activity is that of travelling in a motor car then the controller remains in this state until the accelerometer based activity recognition method detects that the carrier of the cellular phone 1 is no longer travelling in a motor car. At this point the controller will then ascertain whether the carrier of the cellular phone 1 is walking S8. If the controller ascertains that the carrier of the cellular phone 1 is walking then the controller remains in this state until the accelerometer based activity recognition method detects that the carrier of the cellular phone 1 is no longer walking. At this point the controller stops all activity recognition methods S19 and runs CSF SI .

Fifth Embodiment

In an alternative embodiment the cellular phone 1 does not use activity recognition techniques continuously during a transition between two POIs because the context service uses past user behaviour to predict the POI that a person is likely to transition to and the activity or activities that will be performed by the user during the transition.

In this embodiment the context service running on the cellular phone 1 provides the core POI and Transition functionality including: recognising a person returning to previously visited POIs, automatically creating identifiers for new (previously unknown) POIs, identifying transitions between POIs. In addition to this the context service running on the cellular phone 1 will also record (persist to storage on the cellular phone 1) data relating to POIs, transitions and a person's movements throughout this POI network. This includes the following: a person visiting a POI (arrival time and date, duration, departure time), transitions (departing from POI and arriving at POI, activities performed during the transition e.g. walking, driving etc., the time and date, duration). This information is referred to as the Historical data.

In the embodiment the context service uses the Historical data relating to transitions between POIs to predict the POI that a user is likely to be transitioning to. For example, a person leaving their home at 8am on a Monday morning is more likely to be headed to work than if they were leaving their home at 8pm on a Monday evening. If the context service predicts the target POI with a high degree of confidence then the use of the activity recognition techniques described in embodiments 1-4 can be reduced. This behaviour is implemented as follows. The person leaves a POI, commencing a transition to another POI. Upon recognising that the person has left a POI and started a transition the context service makes a prediction as to the POI that the user is likely to be transitioning to. The prediction is based upon the person's past behaviour. If the prediction made by the context service is scored with a high degree of certainty (the context service believes that the prediction is highly-likely to be accurate) then an optimised approach to activity recognition is deployed. In this embodiment this involves periodically switching on the activity recognition service to confirm that the user is still performing the predicted activity. In this embodiment a 2 minute interval is used. It will be apparent to those skilled in the art that different time periods can be used without departing from the scope of the invention. The activity recognition service used is based on the predicted activity e.g. if walking is to be sensed then an accelerometer service or CSF will be used. This process of periodically confirming activity is repeated until either the person arrives at the predicted POI or the predicted time that the transition will take is exceeded. If the predicted time is exceeded continuous activity recognition will then be started. Sixth Embodiment

In an alternative embodiment the cellular phone 1 does not use activity recognition techniques during a transition between two POIs because the context service uses past user behaviour to predict the POI that a person is likely to transition to and the activity or activities that will be performed by the user during the transition.

This embodiment is similar to the fifth embodiment in that the context service is required to make a prediction of the target POI that a person will transition to. The primary difference is that no activity recognition services are used whilst the user is transitioning between POIs unless the predict time that the transition should take is exceeded. This behaviour is implemented as follows. The person leaves a POI, commencing a transition to another POI. Upon recognising that the person has left a POI and started a transition the context service makes a prediction as to the POI that the user is likely to be transitioning to. The prediction is based upon the person's past behaviour. If the prediction made by the context service is scored with a high degree of certainty (the context service believes that the prediction is highly-likely to be accurate) then no activity recognition services are used for the predicted duration of the transition. If the predicted time is exceeded continuous activity recognition will then be started.

Modifications

A particular advantage of the invention described in the above embodiments is that power consumption is minimised. Turning off the power to a GPS receiver module when it is not needed results in the more efficient use of stored power. This extends the time duration before the battery needs to be recharged. It will be apparent to those skilled in the art that the behaviour of the controller could be optimised to suit other scenarios. For example, if the controller uses the clock on the cellular phone 1 and ascertains that it is night time then the controller could even minimise the use of CSF, perhaps running it every ten minutes to confirm the cellular phone 1 is still not moving. Other profile changes may be based upon whether the cellular phone 1 is being held in the users hand, whether the cellular phone is inside or outside, whether it is currently at the users home or at their office.

Although the embodiments described above relate to a GSM network, the 'handoff behaviour (or 'handover' behaviour, as it is also known) exists in all cellular networks due to the need to reuse frequencies. Hence this invention will work with CDMA, UMTS and other types of cellular networks.

It will be apparent to those skilled in the relevant art that the methods described above can be implemented as software programs configured to cause a processor or processor system to implement the method. For example, the methods may be implemented as software applications to be installed and run on a cell phone. Equally, the methods may be implemented in hardware, in firmware loaded onto a device such as a Read-Only Memory, or on a specially configured IC, such as an ASIC.

Although the invention has been described above with reference to one or more preferred embodiments, it will be appreciated that various changes or modifications may be made without departing from the scope of the invention.

It will be apparent to those skilled in the relevant art that although the above embodiment has been described using the Global Positioning System (GPS) and GPS receivers, any type of satellite positioning system such as the proposed European system Galileo could be used and the invention is not limited to GPS.

Claims

1. A method of determining a context of a mobile device, the method comprising: when the mobile device is stationary, determining that the mobile device is at a place of interest;

when the mobile device is not stationary, determining that the mobile device is travelling; and

determining a mode of travel,

wherein the step of determining the mode of travel comprises:

selecting an activity recognition method from a plurality of available activity recognition methods; and

determining the mode of travel using the selected activity recognition method.

2. A method as claimed in claim 1, wherein one of the plurality of available activity recognition methods comprises detecting signals transmitted from wireless beacons.

3. A method as claimed in claim 2, wherein the step of detecting signals transmitted from wireless beacons comprises detecting signals transmitted from base stations of a cellular telephone network.

4. A method as claimed in claim 3, wherein the cellular telephone network is a GSM network.

5. A method as claimed in claim 3, wherein the cellular telephone network is a UMTS network.

6. A method as claimed in any of claims 3 to 5, wherein the step of determining the mode of travel, using the activity recognition method that comprises detecting signals transmitted from wireless beacons, comprises detecting a number of cells from which signals can be detected.

7. A method as claimed in any of claims 3 to 6, wherein the step of determining the mode of travel, using the activity recognition method that comprises detecting signals transmitted from wireless beacons, comprises detecting signal strengths of the signals that can be detected.

8. A method as claimed in any of claims 2 to 7, wherein the step of selecting an activity recognition method comprises:

determining the mode of travel using the activity recognition method that comprises detecting signals transmitted from wireless beacons; and

when the activity recognition method that comprises detecting signals transmitted from wireless beacons indicates that the mode of travel has changed, selecting another activity recognition method from said plurality of available activity recognition methods.

9. A method as claimed in any preceding claim, wherein one of the plurality of available activity recognition methods comprises using a satellite positioning system to determine a position of the mobile device.

10. A method as claimed in claim 9, wherein the satellite positioning system is the GPS system.

11. A method as claimed in any preceding claim, wherein one of the plurality of available activity recognition methods comprises using at least one accelerometer to detect movement of the mobile device.

12. A method as claimed in any preceding claim, comprising:

on determining that the mobile device is travelling, predicting a place of interest to which the mobile device is travelling; and

performing the step of determining the mode of travel only if said prediction has a low degree of confidence.

13. A method as claimed in claim 12, further comprising: periodically determining the mode of travel of the mobile device while the mobile device is travelling.

14. A method as claimed in claim 12 or 13, wherein the step of predicting a place of interest to which the mobile device is travelling comprises predicting a travelling time to the predicted place of interest, the method further comprising:

performing the step of determining the mode of travel if the mobile device is travelling for longer than the predicted travelling time.

15. A mobile device, being adapted to determine its context by means of a method according to any of claims 1 to 14.

16. A computer software product, comprising computer readable code, containing instructions for causing a device to determine a context of a mobile device by means of a method according to any of claims 1 to 14.