WO2007080413A1

WO2007080413A1 - Search platform

Info

Publication number: WO2007080413A1
Application number: PCT/GB2007/000091
Authority: WO
Inventors: Jiazhen Wu; Fred Bierhaus; Massimo Vitali
Original assignee: Vodafone Group Plc
Priority date: 2006-01-13
Filing date: 2007-01-12
Publication date: 2007-07-19
Also published as: GB2434230A; JP2009523284A; GB0600678D0; GB0700662D0; EP1982275A1; CN101401099A

Abstract

In providing a network search platform for mobile devices, the search platform can be adapted to tailor the results of a search to the particular subscriber, mobile device and/or search context (date, time, location, profile etc.). By mandating that content be accompanied by associated meta information, and by providing a local user agent that understands the formatting of said meta information, a search engine can assemble a deep-indexed content database. As a result, search results representing content items that are most suitable for the subscriber can be presented while minimising the chances of delivering unwanted content.

Description

SEARCH PLATFORM

The invention relates to a search platform for mobile devices. In particular, the invention relates to a search platform adapted to tailor the results of a search to the particular subscriber, mobile device and/or search context (date, time, location, profile etc.)

Search engines have become central to a rich experience of the content stored in computing networks. Along with directory portals, search engines facilitate effective use of the internet (especially the Web).

Search engines do however present their own challenges. Often, a search request will give rise to many more search "hits" in the search results than can reasonably be assessed. The majority of search engines limit the number of browsable results to a more manageable figure and attempt to identify the most relevant hits and rank those hits ahead of otherwise similar hits. This problem is often compounded in the mobile environment, where there is generally far less browsable screen area and where charging models can mean that the download of each byte of data represents a monetary cost to the user as well as a corresponding download delay.

One conventional way of managing the vast body of potential search hits is to filter results before rendering the search results, thereby reducing the number of hits presented to the terminal user. This technique is not unique to the mobile environment.

Another challenge to the provision of mobile search services on mobile terminals is the proliferation of different devices, operating systems and standards in the mobile arena. Content that is browsable in one device can be unusable in another, but this may not be apparent to the terminal user until the content has been downloaded - leading to a poor user experience and discouraging further attempts at download.

In an attempt to address these difficulties, presentation format standards have been developed specifically with presentation on the more limited display screens of mobile devices in mind. One such standard, wireless access protocol (WAP), has become widely adopted by mobile network operators when providing browsable content to mobile devices. This protocol has an associated content format, wireless markup language (WML). The WML specification is based on XML.

WML documents ("decks") are stored on a web server in much the same way as HTML (Hyper Text Markup Language) documents. Mobile terminals access WML decks via a WAP gateway. WAP gateways provide an interface with WML content, and increasingly with the World Wide Web.

It is known to "transcode" or process webpages not originally formatted for presentation on mobile terminals into a format suitable for display on such terminals (for example, mobile phones and personal digital assistants, PDAs).

This transcoding process is hidden from the terminal and providing the network operator does not prevent it, allows the terminal to access transcoded pages in the same way that a PC browser application accesses HTML content, using a URL (universal resource locator). Transcoding is not always necessary since selected terminals (having more advanced processing capabilities) actually support the more processor intensive formats, XHTML and/or HTML. Clearly, in the mobile environment, the importance of generating search results that more accurately reflect the capabilities and profile of the user (and his device) is paramount.

A known first step in the provision of mobile search functionality is to filter content based on whether the device supports WML or XHTML before presenting the results of a search to a user.

More recently, attempts have been made to provide device specific search capability. Such search capabilities are limited in their effectiveness as they rely on accurate matching of device to content. Conventional device specific search tools are by no means comprehensive, in part because of the difficulty of maintaining an up-to-date record of which content is suitable for which device.

Furthermore, they have to integrate individual content feed from each content provider, which is often labour intensive.

It is also known to allow certain "hits" to become ranked out of their "natural" order by "boosting" the ranking as a matter of policy, (as may arise when accepting advertising sponsorship for listings on mobile search results pages or encouraging users to obtain content from within the network provided by the supplier or their affiliates).

It is therefore an objective of the invention to overcome the above mentioned limitations of existing mobile search platforms.

According to one aspect of the invention, there is provided a method for generating search results from an index of a content store containing a plurality of content items over a wireless telecommunications connection, each content item having corresponding meta information associated therewith, the meta information comprising at least one of device, subscriber and context specific information, the method comprising: receiving a search request message from a user using a mobile device, the search request message including user search terms; obtaining contextual data representative of the mobile device, user subscription and network capability; initiating a search of the content store index in accordance with the user search terms; determining, for at least one of the user search terms, whether there exists one or more correlated index entry in the content index; and where a correlation exists, generating search results, the results of the search including respective links to correlated content items ranked in accordance with the contextual data.

In addition to filtering on device specific criteria, the method may include a step of filtering based upon contextual criteria, for example: network capability (e.g. is the bearer technology GPRS, EDGE, UMTS/3G, or HSDPA etc.? what is the current network load status?) and user profile (whether automatically and dynamically generated or manually provided by the user).

The user profile may include an indication of location. This location information may be manually entered by the subscriber or alternatively acquired through an automatic location mechanism: for instance using Cell ID or global positioning satellite (GPS) information. Bothe GPS and Cell ID techniques are well know in the field and will not be detailed in the present context.

The invention thus provides a deep link device-specific search with categorised result display. By leveraging user information, subscription information, device information as well as the search terms input in any search request, the effectiveness of the mobile search experience is significantly enhanced. Mobile search as described here improves on the user experience from search engines such as Google and Yahoo by returning search results that are based on a detailed context profile of the device, user, and network capability. The mobile user is no longer presented with content, often chargeable, downloadable digital content, which is generally not optimal for his/her specific device and circumstances.

Search results are conveniently tailored to reflect whether a user is in the right mobile network coverage area (e.g. 3G) to receive the content.

Content "on net" is required to carry additional metatags that indicate deep information about the content: e.g. whether the content is for a specific device, for colour displays only, for subscribers to a paid for channel only or indeed for "adults only". When the crawler agents generate their indexed database (a content store index), they provide a richer database for a corresponding "intelligent" search engine to take advantage of. The results are then ranked in such a way that the most relevant content/results are given higher weighting.

For a better understanding of the present invention, embodiments will now be described by way of example, with reference to the accompanying drawings, in which:

Figure 1 shows the architecture of a typical mobile portal including a mobile search platform in accordance with the present invention;

Figure 2 shows the operation of the mobile search platform of Figure 1 in more detail;

Figure 3 shows the hierarchical structure of crawler functionality; and Figure 4 shows the general MVC (Model/View/Controller) architecture of the content assembly engine (CAE) portion of the mobile search portal.

Figure 1 shows the architecture of a typical mobile portal system including a mobile search platform 100,110,120. The operation of the components of the platform is now described:

Content is stored in one or more content stores 150. Each content item has associated meta information which describes a range of attributes of the content item. One standardised way of doing this is to stipulate the use of RDF

(Resource Descriptor Framework) metadata. RDF is a W3C standard that is defined as a language to describe the information of the World Wide Web resources and their specific metadata (e.g. title, author, creation date, date of last modify, copyright, licensing and others) and "objects" that can be identified on the network.

The W3C Metadata Activity webpage describes RDF as follows: Resource Description Framework (RDF) "provides a more general treatment of metadata".

RDF is a declarative language and provides a standard way for using XML to represent metadata in the form of statements about properties and relationships of items on the Web. Such items, known as resources, can be almost anything, provided it has a Web address. This means that you can associate metadata with a Web page, a graphic, an audio file, a movie clip, and so on.

RDF provides a framework in which independent communities can develop vocabularies that suit their specific needs and share vocabularies with other communities. In order to share vocabularies, the meaning of the terms must be spelled out in detail. The descriptions of these vocabulary sets are called RDF Schemas.

A schema defines the meaning, characteristics, and relationships of a set of properties, and this may include constraints on potential values and the inheritance of properties from other schemas. The RDF language allows each document containing metadata to clarify which vocabulary is being used by assigning each vocabulary a Web address. In short, RDF is extensible.

In the embodiment illustrated in Fig 1, RDF schema are defined to describe meta information about mobile content, such as ringtone, game, picture and music. The meta information may include indications of whether a particular content item is suitable for a given class of mobile device.

The search index used in Fig 1 is hosted by a remote external partner. Rather than populating the search index 120 by crawling the content store 150 from the remote site of the search index 120, the Applicant has developed an "internal search crawler" 160, based on a user agent that provides:

• the ability to have different sections of the content store crawled at different frequencies

• a better ability to monitor search crawl progress

• improved support for RDF meta-data

• support for a single RDF meta-data file that contains information on multiple content items.

As a result the internal search crawler 160 delivers improved quality and number of relevant search results.

Facilitating the introduction of the internal search crawler 160 has necessitated some minor but essential differences in the detailed requirements of the meta- tagging carried out by partner content providers (so-called partner markup language or PML) and RDF file syntax. In particular, the user agent upon which the internal crawler 160 is based is given access all content for all handsets in addition to all corresponding RDF files.

The internal crawler 160 is arranged to parse multi-entry RDF files, where they appear, giving content providers 140 the opportunity to modify their RDF creation logic to take advantage of this.

Logic is introduced in the internal search crawler user agent 160 to ensure that if the URL of the referring page matches the URL embedded within the RDF file, results will be merged to single search index entry, thereby improving the content finding capability.

The user agent may be arranged to use relative links in meta-refs to RDF, and in the RDF files themselves.

For ease of use, certain limitations may be mandated with respect to the RDF files generated for content. For instance: • the rdf "about" attribute must be presented in the format <rdf:Descriρtion about="some/link"> and NOT in the format <rdf:Description rdf : about="some/link">

• RDF files must contain a dc: description attribute

• RDF files must have suffix ".xml" (query strings allowed: eg /showrdf.xml?id=1234 )

• RDF files must be sent as type text/xml

Internal crawler execution

The internal crawler 160 requests a page, which is specified as an entry point in the configuration file, to the web server after identifying itself. Once the page is obtained, it reads and processes the content of the RDF meta information about the content, along with 3rd party supplied pages, extracting title, text, links and all other relevant information. The product follows new links found in the current page while keeping an account of the list of links that have already been followed. The configuration file also has a configurable filter that allows limited sites navigation using a pattern (i.e. a string). Each time the product crawls a page the valid information obtained is stored in a local database (210 in Figure 2).

Different geographic territories within a given mobile telecommunications network operator's network are often served by corresponding operating companies. Each operating company may choose to install a local version of the software stack for handling the provision of content via a terminal application. A bespoke internal crawler can be implemented in each operating company's service delivery platform (or "core stack").

As each internal crawler system can be arranged to run within software stacks appropriate to each operating company, there is an immediate speed advantage over conventional externally provided crawler systems: • the crawl happens at the assembly layer, meaning that there is no requirement to render content for the internal crawler 160 • the incoming crawl requests are immune to Internet latency

By having the internal crawler 160 under the control of the network service provider (or operating company) rather than a third party, multiple crawler tasks can be set running at different times and frequencies, allowing each local operator to see in real-time what is happening. As a result: frequently changing content (for example - news stories) can be crawled more frequently than relatively static content (for example - ringtones); new or modified existing crawler start points can be added very much quicker in response to requests from content providers; and the facility for monitoring and debugging crawler progress can be enhanced to identify and fix problems that stop content items being indexed.

Figure 3 illustrates the software hierarchy of the internal crawler application 160, whereby the frequency and timing of crawling tasks can be scheduled. A varying number of individual instances of the agents ("threads") can operate upon any given task. Each crawling session can in turn mandate a plurality of different tasks.

The internal crawler 160 has been designed to work not only with multiple small RDF meta-data files that reference a single piece of content (an arrangement known from the conventional FAST-SEARCH crawler from FAST[RTM]), but also with large RDF meta-data files that reference multiple items of content ("multi-entry RDF files"). Consequently, for content providers 140 who store details of their content in a database 150, if large RDF files are created (or programmed to be created on-the-fly by server-side logic) with details of all content within it, then the information in those large RDF files will be inserted directly into the index 120, with no need to crawl furtfier pages from that provider. Furthermore, content from third party content providers 140 will become much more visible in the search results, driving increased user traffic and revenue.

The operation of the search platform and particularly of the Search Content Push Engine (SCPS) 180 is illustrated in Figure 2. The SCP Engine 180 has a number of execution functionalities, including: XMLwriter, Uploader and cleaner functionalities. These functionalities are described below. XMLwriter execution

Each single crawling task populates a local database 210 (e.g. an Oracle [RTM] database) with crawling information. These need to be transformed into XML data in order to be uploaded onto the remote search index 120 hosted by the external search engine 110. The XMLwriter retrieves task data (i.e. crawling information) that are stored in the database and converts them into customized XML files accordingly to specifics of the remote index database 120.

Uploader execution The XML files generated by the XMLwriter are uploaded into the remote search index 120 as documents either replacing old versions of the same documents, or inserting new documents if no old ones exist. The data can then be viewed through the mobile interface 130.

Cleaner execution

The Cleaner establishes a connection to the remote search index and removes a set of documents having specific features defined by the settings that appear in the apposite configuration file. Several different cleaning types are supported, starting from the removal of the obsolete documents and ending with the removal of all the documents having a specific field value.

The operation of the mobile search portal 100 is perhaps best understood when one considers the illustrative use case of a subscriber choosing to search for content via the portal 100.

The portal 100 presents a search interface 130 to the subscriber's mobile device, which allows the subscriber to specify search criteria. Let us assume that in this case the subscriber enters a word or phrase. The search interface 130 may further provide further options. Advantageously, there may be an option to limit the search based on content type. There may also be an option to require that all content types are searched. When the content type limitation option is not taken, it may be assumed that the subscriber wishes to 5 search all available content. If however the subscriber chooses a content type, the portal 100 allows the subscriber to initiate a search.

The system receives the search request from the subscriber and processes the portal request. Firstly, the subscriber is authenticated. Provided the subscriber is 0 authenticated, the request is authorized.

In the authentication process, contextual information is obtained. This contextual information includes information relating to the subscriber profile of the user (e.g. language preference, safe-search criterion, credit level etc) and information 5 delivered by the mobile terminal in the authentication process (e.g. subscriber's device type).

The portal system 100 then checks the submitted search criteria to ensure that the search is valid (e.g. it contains at least two recognised characters). The system ZO then sends the search criteria along with the contextual information and content type (if selected) to the search engine 110.

The search engine 110 processes valid search criteria and generates ranked, device- specific search results. >5

The core stack of the portal system 132 then receives the device specific search results from the search engine 110 and renders them for presentation on the subscriber's display. The core stack (or service delivery platform) 132 includes a third party integration component (3PI), 230 a content (or common) assembly engine (CAE) 240 and a content rendering engine (CRE) 250. These components are shown in both Fig 1 and Fig 2, but only given reference numbers in Fig 2.

The 3PI 230 receives RDF tag information in partner markup langage (PML) files provided by content providers 220, 222, 224 in addition to the content itself. The 3PI 230 extracts the RDF tag information corresponding to the content items and supplies the extracted information as "fragments" of operator mark-up language (in the present applicant's case: Vodafone Content Markup Language or VCML).

The operator markup language fragments are then assembled at the assembly layer for delivery to a higher level, rendering layer. The functionality of the assembly layer is provided by the CAE 240, a software component that interprets incoming requests from users, obtains information from content sources (i.e. the extracted RDF tag information in the VCML fragments), and assembles the pieces that will make up the response into a single content document (in this instance, a VCML "page"). It then sends the assembled content to the rendering layer.

The general architecture of the CAE 240 is shown in Figure 4. Within this so- called MVC (Model/View/Controller) architecture, business logic classes and beans comprise the Model, JSP files and custom tags comprise the View, and Actions comprise the Controller. The business logic can be further separated into several layers: Service objects and Manager objects. The Service objects are the core of the business logic and sit on top of Manager objects, which encapsulate lower level functionality, such as persistence, or, in the case of the Search Portal, connections to the external search servers. A key example of an Action is the SubmitSearchAction, which is responsible for extracting the user's query from the request, validating it, and calling the business logic to perform the search. To complement this Action, the present CAE provides SearchService and SearchManager. SearchService is responsible for search engine implementation independent business logic, allowing such things as the boosting of internal results. SearchManager is responsible for business logic which is specific to the implemented search engine, such as generating a syntactically correct query and transforming search engine result objects into generic value objects for consumption by the higher layers.

The content rendering layer receives search result objects, parses the markup language and delivers the result object in a format suitable for display. Using information extracted at authentication of the user device, the CRE 250 recognises an end-user's device, and uses the attributes of the device to convert resources such as images and text coming from the CAE 240 to a size and format appropriate for the particular device. It also manages the connection between the user device and the rest of the Service Delivery Platform: in particular the provision of the search interface 130 for extracting user requests.

The core stack 132 may be provided with a spelling inference engine (not shown) for generating an alternative spelling response along with the search results. This addresses the common experience of misspelling, suggesting a "most likely" alternative search term.

By insisting upon authorization of the request against the subscriber profile, the portal 100 can filter out results corresponding to age-classified and/or age- restricted content.

Since certain content items have an associated monetary value, the portal 100 is arranged to detect a Service ID tag (i.e. a charging tag) within the links to content items returned. If a Service ID is contained within an item returned by the search engine 110 and the subscriber chooses to download that content, a charge will be deducted from their account.

If no Service ID is contained within an item returned by the search engine 110, the portal displays the item in a search results deck. The deck shows the entered search criteria, the total number of results matching the search criteria to be displayed, and each search result as a hyperlink, with a teaser description for each search result. The search results may be displayed grouped by content type. The search results may distinguish between content that is hosted by the service provider ("on net") and content hosted on web servers not controlled by the provider ("off net"). The results deck may show this separation by displaying the total number of results for each group as a distinct item.

Advantageously, the search results are displayed within each group ordered according to relevancy. Thus, the system can be configured to display results with a relevancy ranking equal to or greater than a predetermined threshold. "On net" search results may be "boosted" individually above similar "off net" results or they may be presented as an "on net" group above a corresponding "off net" group. Where a charge is associated with download of content, the search result deck may indicate the fact visually, for example with an icon.

The search process outlined above can be supplemented by a "search tips" facility, whereby the subscriber may request guidance about suitable search terms, wildcard symbols.

To narrow the search process, the subscriber may qualify a word search with a content type limitation. More generally he may be taken through a search process based on the context of the search request, a so-called contextual search. The subscriber chooses to search for content from a contextual search hyperlink. The system returns a search dialogue page with the context (e.g. content type) pre-selected. Examples of content types are ringtones, games, pictures & music. The system may also allow the subscriber to specify further contextual search criteria.

Using the pre-filled search dialogue page, the subscriber enters a word or phrase for their contextual search. The system handles this contextual search by initiating a search within the defined context (content type).

Similarly to searching by context, the subscriber may search within distinct "content channels" (such as a news broadcast feed or a chart music/video content channel). The subscriber chooses the channel search option from a 3^rd party brokered partner page. The system allows the subscriber to specify the channel search criteria and returns a pre-filled search dialogue page: pre-filled on this occasion with a predetermined channel selected. Again, the subscriber uses the pre-filled search dialogue page to enter a word or phrase for a channel search and the system responds to the entry of search terms in this dialogue page by initiating a corresponding search within the selected channel for the subscriber.

Service ID information can be used to refine further the results of a search task. In many cases, the raw results of a search task include service ID information. Where service ID information is present in a content item returned by the search engine, the system may send a request for service related information, such as pricing data, to an appropriate service application (e.g. a charging and pricing application), not shown. The request is arranged to contain the service ID information for the content item. The system receives a corresponding response from the service application. The information displayed by the system can then be tailored to reflect the service related information corresponding to the content item in the search results. Thus, if the content item required subscription, selecting the link to the content item would lead to a content subscription request page instead of a direct link to the actual content.

The system may be arranged to return a suitable error message when it determines that the search criteria entered are invalid.

Likewise, the system may be arranged to display a message advising the subscriber that no results were found when it receives a "no results found" response from the search engine. The cause of a null result may be misspelling. In addition, the occurrence of a "no results" response may be the trigger for an update handling mechanism. Whenever the "no results" response is received the mechanism is capable of checking the context of the request and in certain circumstances (e.g. the device or firmware being used by the subscriber is unrecognised) may suggest or trigger an update check. As a result the occurrence of such "no results" responses can be reduced.

Advantageously, the system may be provided with a spelling engine, so that if the "no results found" response contains an alternative spelling, the system displays an alternative spelling message with the alternative spelling displayed as a search hyperlink.

The spelling engine may equally well be used, when results are found. The system may display a message advising the Subscriber that an alternative spelling has been found and the alternative spelling may be displayed as a search hyperlink. Furthermore, in view of the difficulty of accurate text entry using mobile handsets, the user may actively request the application of the spelling engine to verify their typed search criteria.

Where no search engine response is received within a pre-defined timeout period, the system logs a communication failure event and displays an error message. A determination that search engine is unavailable is met with a similar result. Again the system logs a communication failure event and displays an appropriate error message.

The mobile search platform thus filters search results intelligently, taking account of:

• device features (e.g. full track music capable phone), device type (e.g. Advanced phone with wide screen), device name and firmware installed on device

• bearer network capability at the time (for example, whether the network is analogue or digital, GPRS (2.5G), EDGE or 3G)

• user profiles (e.g. child, 18+ etc)

• user subscription to the content

Clearly, there may be mismatches between the device capabilities claimed by manufacturers and the verified, empirical capabilities of a device. A database of empirical capabilities may be addressed in preference to using manufacturer specifications to determine whether a given device will give satisfactory results when paired with a particular content item.

The intelligent crawling solution described above provides crawling of the mobile content using both a proprietary content format (in the case of the Applicant - Vodafone Content Markup Language, VCML) and mobile RDF (Resource Description Framework) tagging scheme. The intelligent crawler automatically associates content with their meta-information such as device specific features, network suitability, user profile suitability, subscriptions, categories of the content, ranking boosting value of the content etc. This makes rendering search results simple yet relevant to the personal context of the user.

Claims

1. A method for generating search results from an index (120) of a content

store (150) containing a plurality of content items over a wireless

telecommunications connection, each content item having corresponding meta

information associated therewith, the meta information comprising at least one of

device, subscriber and context specific information, the method comprising:

receiving a search request message from a user using a mobile device

(130), the search request message including user search terms;

obtaining contextual data representative of the mobile device (130), user

subscription and network capability;

initiating a search of the content store index (120) in accordance with the

user search terms;

determining, for at least one of the user search terms, whether there exists

one or more correlated index entry in the content store index (120); and

where a correlation exists, generating search results, the results of the

search including respective links to correlated content items ranked in accordance

with the contextual data.

2. A method as claimed in claim 1, further including a step of filtering based upon contextual criteria and user profile.

3. A method as claimed in claim 2, wherein the user profile is automatically and dynamically generated from information obtained in establishing the telecommunication connection.

4. A method as claimed in claim 2 or claim 3, wherein at least a portion of the user profile is manually provided by the user during the telecommunication connection.

5. A method as claimed in claim 4, wherein the user location is manually input by the user during the telecommunication connection.

6. A method as claimed in any one of the preceding claims, wherein the results of the search are tailored to reflect whether a user currently has access to a bearer technology appropriate for transporting the content.

7. A method as claimed in any one of the preceding claims, wherein each content item is required to carry additional meta information, whereby the content index provides indexing of the additional meta information in addition to the meta information.