US20040264677A1 - Ideal transfer of call handling from automated systems to human operators based on forecasts of automation efficacy and operator load - Google Patents
Ideal transfer of call handling from automated systems to human operators based on forecasts of automation efficacy and operator load Download PDFInfo
- Publication number
- US20040264677A1 US20040264677A1 US10/610,486 US61048603A US2004264677A1 US 20040264677 A1 US20040264677 A1 US 20040264677A1 US 61048603 A US61048603 A US 61048603A US 2004264677 A1 US2004264677 A1 US 2004264677A1
- Authority
- US
- United States
- Prior art keywords
- call
- automated
- dialog
- human operator
- xfer
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Abandoned
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04M—TELEPHONIC COMMUNICATION
- H04M3/00—Automatic or semi-automatic exchanges
- H04M3/42—Systems providing special services or facilities to subscribers
- H04M3/50—Centralised arrangements for answering calls; Centralised arrangements for recording messages for absent or busy subscribers ; Centralised arrangements for recording messages
- H04M3/51—Centralised call answering arrangements requiring operator intervention, e.g. call or contact centers for telemarketing
- H04M3/523—Centralised call answering arrangements requiring operator intervention, e.g. call or contact centers for telemarketing with call distribution or queueing
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04Q—SELECTING
- H04Q3/00—Selecting arrangements
- H04Q3/64—Distributing or queueing
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS OR SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING; SPEECH OR AUDIO CODING OR DECODING
- G10L15/00—Speech recognition
- G10L15/26—Speech to text systems
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04M—TELEPHONIC COMMUNICATION
- H04M3/00—Automatic or semi-automatic exchanges
- H04M3/42—Systems providing special services or facilities to subscribers
- H04M3/50—Centralised arrangements for answering calls; Centralised arrangements for recording messages for absent or busy subscribers ; Centralised arrangements for recording messages
- H04M3/527—Centralised call answering arrangements not requiring operator intervention
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04M—TELEPHONIC COMMUNICATION
- H04M2201/00—Electronic components, circuits, software, systems or apparatus used in telephone systems
- H04M2201/14—Delay circuits; Timers
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04M—TELEPHONIC COMMUNICATION
- H04M3/00—Automatic or semi-automatic exchanges
- H04M3/42—Systems providing special services or facilities to subscribers
- H04M3/54—Arrangements for diverting calls for one subscriber to another predetermined subscriber
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04Q—SELECTING
- H04Q2213/00—Indexing scheme relating to selecting arrangements in general and for multiplex systems
- H04Q2213/13072—Sequence circuits for call signaling, ACD systems
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04Q—SELECTING
- H04Q2213/00—Indexing scheme relating to selecting arrangements in general and for multiplex systems
- H04Q2213/1337—Operator, emergency services
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04Q—SELECTING
- H04Q2213/00—Indexing scheme relating to selecting arrangements in general and for multiplex systems
- H04Q2213/13378—Speech recognition, speech analysis
Definitions
- the present invention relates generally to systems and methods that facilitate communications between devices, systems, processes, and/or individuals. More particularly, the present invention relates to employment of dynamic policies for transferring people from an automated call routing system to a human operator given the likelihood of success, the effort associated with continuing to interact with an automated system and the wait times associated with entering a queue for a human operator.
- an automated call routing system may field all internal directory assistance calls. Using speech recognition, the system attempts to uniquely identify one of over thousands of possible name entries in a company's global address book. Needless to say, a vocabulary of this size is bound to have deleterious effects on the performance of the speech engine.
- the present invention relates to guiding calls in accordance with an automated call routing system.
- One or more decision models are provided that output policies for switching people from an ongoing automated system to a human operator based on context-sensitive analysis of the spoken dialog situation at hand. This also relates to applying decision-theoretic principles to reason about the ideal melding of people and automated reasoning systems.
- callers' experiences with a call routing system are optimized by developing models that consider the probability that callers will be ultimately successful (or not) in their collaboration with the automated system, the expected number of steps required for that success, and the current load on human personnel.
- a voice routing system for directory assistance can be analyzed via data logs of a system's performance.
- One or more probabilistic models can then be constructed from the data logs regarding the likelihood of success or failure with the automated system, and the number of steps, overall duration, effort, or frustration with the use of the automated system, given that there will be eventual success of the interaction.
- the present invention couples path-dependent probabilities from learned models with a decision analysis to optimally guide the transfer of calls to a human operator, given information about the expected current wait times associated with a transfer to a human operator.
- FIG. 1 is a schematic block diagram illustrating an automated call routing and decision system in accordance with an aspect of the present invention.
- FIG. 2 is a diagram illustrating call routing statistical data in accordance with an aspect of the present invention.
- FIG. 3 is a diagram illustrating dialog features in accordance with an aspect of the present invention.
- FIG. 4 is diagram illustrating a probability tree in accordance with an aspect of the present invention.
- FIG. 5 is a diagram illustrating a dependency network for dialog features in accordance with an aspect of the present invention.
- FIG. 6 is a diagram illustrating a Markov Dependency network in accordance with an aspect of the present invention.
- FIG. 7 is a prior distribution of an expected number of dialog steps in accordance with an aspect of the present invention.
- FIG. 8 is a flow diagram illustrating systematic automated call routing according to an aspect of the present invention.
- FIG. 9 is a schematic block diagram illustrating a suitable operating environment in accordance with an aspect of the present invention.
- FIG. 10 is a schematic block diagram of a sample-computing environment with which the present invention can interact.
- the subject invention employs decision theory in connection with managing calls. More particularly, management of call traffic and handling of specific calls is important to many businesses, and the subject invention employs decision theory in connection with optimizing handling of calls.
- An automatic call answering system can be employed until the cost associated therewith (e.g., customer hanging up) outweighs the benefits (e.g., minimizing human intervention to deal with the call).
- a system in accordance with the invention employs a decision-theoretic based framework to take a most appropriate action (e.g., use automatic call answering system, end use of call answering system, switch to human operator, route call to different service . . .
- the subject invention optimizes utilization of resources via employment of decision theory and one more models that are trained from past activity logs.
- a component may be, but is not limited to being, a process running on a processor, a processor, an object, an executable, a thread of execution, a program, and/or a computer.
- a component may be, but is not limited to being, a process running on a processor, a processor, an object, an executable, a thread of execution, a program, and/or a computer.
- an application running on a server and the server can be a component.
- One or more components may reside within a process and/or thread of execution and a component may be localized on one computer and/or distributed between two or more computers.
- the term “inference” refers generally to the process of reasoning about or inferring states of the system, environment, and/or user from a set of observations as captured via events and/or data. Inference can be employed to identify a specific context or action, or can generate a probability distribution over states, for example.
- the inference can be probabilistic; that is, the computation of a probability distribution over states of interest based on a consideration of data and events.
- Inference can also refer to logical inferences, including deterministic techniques employed for composing higher-level events from a set of events and/or data. Such inference results in the construction of new events or actions from a set of observed events and/or stored event data, whether or not the events are correlated in close temporal proximity, and whether the events and data come from one or several event and data sources.
- the system 100 includes an automated call routing system 110 that is routinely employed for providing automated responses 120 to one or more callers 130 .
- These systems include processing components, switching components, electronic directories, and associated software such as speech recognition components for communicating with the callers 130 and routing calls to an identified individual at 140 (e.g., voice command indicating individuals last name). If callers 130 have trouble making a connection with an individual or party, the call routing system 110 can connect to a human operator 144 to provide further assistance to the callers.
- one or more decision models 150 are employed with the call routing system 110 to facilitate efficient operations of the system 100 , provide more efficient coupling between callers and respondents, and mitigate caller frustration when interacting with such systems. In one aspect, this achieved by training the decision models 150 via a data log 160 that has recorded data of past activities and interactions with the call routing system 110 . Output from the decision models 150 is then employed for call routing determinations. Such data includes statistical information such as how often speakers have been found or not found, how often an operator has been requested and so forth wherein this data is described in more detail below with respect to FIG. 2. After the decision models have been trained, the call routing system 100 works in concert with the models to facilitate call routing between callers and individuals to be contacted. As will be described in more detail below, the models 150 can be employed in accordance with dynamic policies relating to the costs and benefits of switching a caller to the human operator, for example.
- a diagram 200 is illustrates statistical data that is analyzed in accordance with an aspect of the present invention. Although various categories of data are depicted by the diagram 200 , it is to be appreciated that other categories of statistics relating to call routing performance can be similarly analyzed (e.g., average amount of time to connect).
- categories of statistics relating to call routing performance can be similarly analyzed (e.g., average amount of time to connect).
- SpeakFound The system finds the name in the directory.
- SpeakNotFound The system infers that the person requested may not be in the directory, and routes the call to an operator.
- OperatorRequest The user requests an operator by pressing ‘0’.
- HelpRequest The user tries to request help by pressing ‘*’ or ‘#’.
- MaxErrrors The system reaches the maximum number of allowed mistakes, and routes the call to an operator.
- NotReady The system is temporarily out of service.
- Undefined The user tries to press a numeric key.
- the diagram 200 depicts an overall breakdown of all individual sessions into their possible outcomes. Due primarily to the large list of employee names in a sample company directory, a call routing system without associated models and policies achieved a dismal success rate of 45%, corresponding to the percentage of SpeakFound outcomes where the system correctly identified the proper name without the user attempting to request help from a human operator. The success rate jumps to 66% when session logs in which the caller did not even attempt to speak a name are removed. The situations where no interaction is attempted disclose the unsettling fact that many callers are likely to completely bypassing interaction with the system. In fact, these “no-name” attempts comprised 31% of the entire data, 85% of all OperatorRequest outcomes and 53% of all HangUp outcomes.
- dialog features 300 are illustrated in accordance with an aspect of the present invention.
- Call routing systems typically treat policies for transferring users to an operator in an ad-hoc manner using handcrafted rules composed of various dialog features, such as the number of questions asked so far in a session.
- the present invention builds probabilistic models from a large database of session logs with the intention of discovering dialog features that were predictive of success or failure.
- the dialog features employed for learning the models fall into four broad categories:
- System and user actions at 310 e.g., asking the user to pick between the top two guesses, the user has pressed a key, etc.
- Session summary features at 320 e.g., number of name attempts detected, overall duration, etc.
- n-best recognitions features at 330 e.g., range of confidence scores assigned by the speech recognition subsystem, mode, greatest consecutive score difference, count of the most frequent first/last/full names, etc.
- Generalized temporal features 340 e.g., number of recurring first names that match in consecutive turns, etc.
- the n-best recognitions features 330 can be derived from a speech recognizer, and the generalized temporal features 340 were included to cover trends between n-best lists. Using varying amounts of feature information, three classes of models (can be more or less than three) were built to estimate the likelihood that a session with the call routing system would eventually end in success or failure.
- a probability tree 400 showing the likelihood of success given any sequence of system actions is illustrated in accordance with an aspect of the present invention.
- a likelihood of success can be determined, that is, p(SpeakFound
- the probabilities can be visually displayed as a tree 400 where each branch represents the system action, as shown in FIG. 4.
- Table 1 below provides an expanded view of several dialog branches extracted from the tree 400 , where n refers to the number of callers reaching that state in the data logs.
- the probability tree 400 the first system action [Operator Intro], which gives a standard prompt but appends the phrase “For an operator, please press ‘0’,” has only a 45% chance of succeeding.
- the system subsequently asks the caller to repeat the entire name and then asks the caller to pick among three possible guesses, the likelihood of success increases to 66%.
- the increased probability of success for this initial interaction is atypical for dialog with the automated system as a whole. Typically, the longer the sequence of actions or path along the tree, the less likely it is that success will be achieved.
- the advantage of the marginal probability tree model 400 is simplicity.
- the data that is required to build the tree 400 is a sequence of system actions for respective sessions and the ultimate outcome. As we will be described in the following section, the sequence of system actions happens to be the most predictive factor in determining success.
- the drawback to this model 400 is that the longer the sequence of actions, or path down the tree, the sparser the set of cases for building robust predictors of the likelihood of success.
- FIG. 5 illustrates a dependency network 500 for dialog features in accordance with an aspect of the present invention.
- another aspect of the present invention employs a Bayesian structure learning to build probabilistic models for predicting session outcome.
- a Bayesian learning tool that performs structure search and model scoring for different predictive models given a data set can be employed to learn a predictive model.
- the WinMine toolkit (Chickering et al. 1997) can be employed to build a dependency network and associated decision tree considering session outcome and other dialog features as input variables.
- the top five dialog features for predicting outcome are displayed in the diagram 500 in order of dependency strength are as follows: (1) the sequence of system actions, (2) the count or number of alternates in the n-best recognitions list, (3) the number of times the user attempted to speak a name, (4) the largest confidence score assigned by the system, and (5) the number of dialog turns—defined as a question-answer pair.
- the dependency model 500 resonates with findings in the probability tree in that the dependency with the strongest link to outcome is the sequence of system actions.
- the first decision split occurs when the system either does or does not identify the proper name after one confirmation attempt. If the confirmation is successful, then the likelihood of a SpeakFound session is almost certain at 99%. Otherwise, the decision tree 500 considers other dialog features to predict success. Consistent with the idea that the longer the sequence of actions, the lower the chance of success, it was found that two of the next five strongest connections were related to the length of the dialog. The remaining three features were parameters output by the speech recognizer.
- Table 2 displays the top five influencing variables for the respective dependency network models. Notice that for the first turn, almost all of the top five variables relate to the n-best recognition list generated by the speech recognizer, including the distribution characteristics of the confidence scores such as skewness and kurtosis. In moving to the second turn, however, generalized temporal features come into play, such as the maximum number of times a first or last name from the first dialog turn shows up again in the second turn. After two turns, the dialog features relate mostly to the length of the dialog.
- FIG. 6 a Markov Dependency network 600 in accordance with an aspect of the present invention.
- temporal dependency networks for Markov pairs of n-best recognition list features were constructed.
- FIG. 6 displays the Markov dependency network 600 . Since the count or number of alternates in the n-best recognition list was consistently selected to be one of the most strongest dependencies, the dependency network shows how it is possible to predict the number of alternates in turn t from n-best list features in turn t ⁇ 1.
- the partial-dialog models outperform the full model as well as the marginal for respective dialog turn datasets with the highest accuracy on the first dialog turn at 94% accuracy.
- the full model also achieves 85% on the complete dataset but provides poorer results when the dataset is decomposed by dialog turn. It is understood that increases in the amount of data can boost the partial models for dialog turn over the marginal models. TABLE 3 Classification accuracy of the marginal, full, and partial-dialog models.
- a goal of the present invention has been to harness the probabilistic models in dynamic decisions about the costs and benefits of shifting a caller to a human operator. Having access to probabilities of the eventual success of a session with an automated system, as a function of the observed state of a dialog, provides a control surface enabling such decision-making.
- probabilistic models with the ability to provide predictions about outcomes provide an immediate way for administrators of automated call routing systems to specify preferences regarding the transfer of callers to a human operator. Such preferences can be represented as a tolerated threshold on failure as a function of the current expected time that callers will have to wait for a human operator, given the current load on operators.
- the probabilistic models can also be employed in call center design. Staffing decisions can be made with the overall system model, constructed by taking into consideration the probabilistic performance of an automated system to route calls successfully, preferences about wait time, characterization of caller volumes, and the time required for addressing callers in a queue waiting for an operator.
- Systems and methods can be provided for handling the overall challenge of optimizing a call routing system design, based on a queue-theoretic formulation.
- a decision-theoretic analysis is provided that minimizes the expected wait time for a caller, given observations about the nature and progress of dialog with an automated system.
- the utility for a user is a function of the number of dialog steps taken so far with the automated system, n, the total additional expected number of steps that will be taken for the current call with an automated routing system, m, and the wait time, w, associated with a transfer to a human operator should a transfer occur. It is noted that, in the general case, not only should the total time be considered, but the nature of the interaction steps. As an example, people may be extremely frustrated with the poor recognitions of the system, even if the overall outcome is accurate. Beyond the number of turns and wait times, the utility of an interaction for a caller may be influenced by other factors. For example, callers may have a negative emotional reaction to working with an automated system versus a human operator. Such factors can be folded into a cost-benefit analysis of routing actions under uncertainty, considering the number and nature of each step in a dialog.
- FIG. 7 is a diagram 700 illustrating prior distribution of the expected number of dialog steps conditioned on the success of the automated call routing system.
- the analysis can be generalized with a conversion of steps to an effective total time of an interaction, where frustration is captured by increases in the effective total time of specific steps.
- the expected number of additional steps can be computed at each point in a dialog for the case where there is eventual success with the use of the automated system, and the case where the automated routing fails for any reason (e.g., the user presses the “O” key to access a human operator) and the user is immediately routed to a human operator.
- FIG. 7 displays the expected number of steps for success at 0 steps—the outset of an interaction.
- the present invention employs p(xfer
- E, ⁇ ) 1 ⁇ p(xfer
- E,xfer, ⁇ ) the probability distribution over the additional number of steps given an eventual failure
- E,success, ⁇ ) the probability distribution over the number of additional steps given a successful interaction with the automated system, and the expected number of steps for each of these conditions, labeled ⁇ m> and ⁇ m′>, respectively.
- the wait time associated with an immediate transfer into the queue for interacting with a human operator is w. This time varies and the current value can be measured or estimated at any moment by measuring the average recent wait times, or by checking the queue of callers waiting for an operator. At the current position in a dialog, the expected wait time associated with continuing the automated interaction versus making an immediate automated transfer to the waiting queue for a human operator can be compared. That is, continue to determine if t a >w. If the expected wait time is greater for continuing to engage the user with the automated system, then transfer the user to the queue for a human operator.
- the present invention can perform different amounts of look ahead, and invoke Equation 1 to compute the expected wait times at points further downstream in the dialog, folding in a consideration of the uncertainty that the user will take different paths conditioned on the current path, and will successfully reach each of the downstream points.
- Employing the decision-theoretic measure promises to minimize the total wait time for users.
- the cost of handling a call with a human operator, C for an organization and take an organizational perspective on decision making can be considered. Rather than just modeling the cost to a user in terms of time and frustration, the following, considers an organization's assignment of the utility as a function the cost of the handling of a call by a human operator, in addition to the organizations assessment of the cost associated with the user's time and frustration,
- FIG. 8 illustrates a methodology for call routing and decision making in accordance the present invention. While, for purposes of simplicity of explanation, the methodology is shown and described as a series of acts, it is to be understood and appreciated that the present invention is not limited by the order of acts, as some acts may, in accordance with the present invention, occur in different orders and/or concurrently with other acts from that shown and described herein. For example, those skilled in the art will understand and appreciate that a methodology could alternatively be represented as a series of interrelated states or events, such as in a state diagram. Moreover, not all illustrated acts may be required to implement a methodology in accordance with the present invention.
- the present invention considers the various interactions and performance of a real-world speech-recognition based on an automated call routing system. As noted above, this can include an analysis of the various system components and operator interactions affecting one or more variables of system performance.
- data is gathered to determine various performance aspects of the system. This can include call answering statistics, routing statistics, success or failure statistics based on various factors such as the amount of time a caller has to wait before being directed into a queue to speak with a human operator, for example.
- one or more probabilistic models can be constructed from the data that is employed to provide probabilities of different paths through the call routing system, including such information as whether an interaction with the automated system will be successful.
- a decision-theoretic analysis of the value of switching to a human operator at different points in a user's interaction with the automated routing system is also provided.
- an exemplary environment 910 for implementing various aspects of the invention includes a computer 912 .
- the computer 912 includes a processing unit 914 , a system memory 916 , and a system bus 918 .
- the system bus 918 couples system components including, but not limited to, the system memory 916 to the processing unit 914 .
- the processing unit 914 can be any of various available processors. Dual microprocessors and other multiprocessor architectures also can be employed as the processing unit 914 .
- the system bus 918 can be any of several types of bus structure(s) including the memory bus or memory controller, a peripheral bus or external bus, and/or a local bus using any variety of available bus architectures including, but not limited to, 11-bit bus, Industrial Standard Architecture (ISA), Micro-Channel Architecture (MSA), Extended ISA (EISA), Intelligent Drive Electronics (IDE), VESA Local Bus (VLB), Peripheral Component Interconnect (PCI), Universal Serial Bus (USB), Advanced Graphics Port (AGP), Personal Computer Memory Card International Association bus (PCMCIA), and Small Computer Systems Interface (SCSI).
- ISA Industrial Standard Architecture
- MSA Micro-Channel Architecture
- EISA Extended ISA
- IDE Intelligent Drive Electronics
- VLB VESA Local Bus
- PCI Peripheral Component Interconnect
- USB Universal Serial Bus
- AGP Advanced Graphics Port
- PCMCIA Personal Computer Memory Card International Association bus
- SCSI Small Computer Systems Interface
- the system memory 916 includes volatile memory 920 and nonvolatile memory 922 .
- the basic input/output system (BIOS) containing the basic routines to transfer information between elements within the computer 912 , such as during start-up, is stored in nonvolatile memory 922 .
- nonvolatile memory 922 can include read only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable ROM (EEPROM), or flash memory.
- Volatile memory 920 includes random access memory (RAM), which acts as external cache memory.
- RAM is available in many forms such as synchronous RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), Synchlink DRAM (SLDRAM), and direct Rambus RAM (DRRAM).
- SRAM synchronous RAM
- DRAM dynamic RAM
- SDRAM synchronous DRAM
- DDR SDRAM double data rate SDRAM
- ESDRAM enhanced SDRAM
- SLDRAM Synchlink DRAM
- DRRAM direct Rambus RAM
- Computer 912 also includes removable/non-removable, volatile/non-volatile computer storage media.
- FIG. 9 illustrates, for example a disk storage 924 .
- Disk storage 924 includes, but is not limited to, devices like a magnetic disk drive, floppy disk drive, tape drive, Jaz drive, Zip drive, LS-100 drive, flash memory card, or memory stick.
- disk storage 924 can include storage media separately or in combination with other storage media including, but not limited to, an optical disk drive such as a compact disk ROM device (CD-ROM), CD recordable drive (CD-R Drive), CD rewritable drive (CD-RW Drive) or a digital versatile disk ROM drive (DVD-ROM).
- CD-ROM compact disk ROM device
- CD-R Drive CD recordable drive
- CD-RW Drive CD rewritable drive
- DVD-ROM digital versatile disk ROM drive
- a removable or non-removable interface is typically used such as interface 926 .
- FIG. 9 describes software that acts as an intermediary between users and the basic computer resources described in suitable operating environment 910 .
- Such software includes an operating system 928 .
- Operating system 928 which can be stored on disk storage 924 , acts to control and allocate resources of the computer system 912 .
- System applications 930 take advantage of the management of resources by operating system 928 through program modules 932 and program data 934 stored either in system memory 916 or on disk storage 924 . It is to be appreciated that the present invention can be implemented with various operating systems or combinations of operating systems.
- a user enters commands or information into the computer 912 through input device(s) 936 .
- Input devices 936 include, but are not limited to, a pointing device such as a mouse, trackball, stylus, touch pad, keyboard, microphone, joystick, game pad, satellite dish, scanner, TV tuner card, digital camera, digital video camera, web camera, and the like. These and other input devices connect to the processing unit 914 through the system bus 918 via interface port(s) 938 .
- Interface port(s) 938 include, for example, a serial port, a parallel port, a game port, and a universal serial bus (USB).
- Output device(s) 940 use some of the same type of ports as input device(s) 936 .
- a USB port may be used to provide input to computer 912 , and to output information from computer 912 to an output device 940 .
- Output adapter 942 is provided to illustrate that there are some output devices 940 like monitors, speakers, and printers, among other output devices 940 , that require special adapters.
- the output adapters 942 include, by way of illustration and not limitation, video and sound cards that provide a means of connection between the output device 940 and the system bus 918 . It should be noted that other devices and/or systems of devices provide both input and output capabilities such as remote computer(s) 944 .
- Computer 912 can operate in a networked environment using logical connections to one or more remote computers, such as remote computer(s) 944 .
- the remote computer(s) 944 can be a personal computer, a server, a router, a network PC, a workstation, a microprocessor based appliance, a peer device or other common network node and the like, and typically includes many or all of the elements described relative to computer 912 .
- only a memory storage device 946 is illustrated with remote computer(s) 944 .
- Remote computer(s) 944 is logically connected to computer 912 through a network interface 948 and then physically connected via communication connection 950 .
- Network interface 948 encompasses communication networks such as local-area networks (LAN) and wide-area networks (WAN).
- LAN technologies include Fiber Distributed Data Interface (FDDI), Copper Distributed Data Interface (CDDI), Ethernet/IEEE 1102.3, Token Ring/IEEE 1102.5 and the like.
- WAN technologies include, but are not limited to, point-to-point links, circuit switching networks like Integrated Services Digital Networks (ISDN) and variations thereon, packet switching networks, and Digital Subscriber Lines (DSL).
- ISDN Integrated Services Digital Networks
- DSL Digital Subscriber Lines
- Communication connection(s) 950 refers to the hardware/software employed to connect the network interface 948 to the bus 918 . While communication connection 950 is shown for illustrative clarity inside computer 912 , it can also be external to computer 912 .
- the hardware/software necessary for connection to the network interface 948 includes, for exemplary purposes only, internal and external technologies such as, modems including regular telephone grade modems, cable modems and DSL modems, ISDN adapters, and Ethernet cards.
- FIG. 10 is a schematic block diagram of a sample-computing environment 1000 with which the present invention can interact.
- the system 1000 includes one or more client(s) 1010 .
- the client(s) 1010 can be hardware and/or software (e.g., threads, processes, computing devices).
- the system 1000 also includes one or more server(s) 1030 .
- the server(s) 1030 can also be hardware and/or software (e.g., threads, processes, computing devices).
- the servers 1030 can house threads to perform transformations by employing the present invention, for example.
- One possible communication between a client 1010 and a server 1030 may be in the form of a data packet adapted to be transmitted between two or more computer processes.
- the system 1000 includes a communication framework 1050 that can be employed to facilitate communications between the client(s) 1010 and the server(s) 1030 .
- the client(s) 1010 are operably connected to one or more client data store(s) 1060 that can be employed to store information local to the client(s) 1010 .
- the server(s) 1030 are operably connected to one or more server data store(s) 1040 that can be employed to store information local to the servers 1030 .
Abstract
The present invention relates to dynamic policies for transferring people from an automated or user-directed call handling system to a human operator, depending on considerations of the likelihood of the failure of the interaction with the call-handling system, predictions about the expected time or frustration associated with using the system, and the current load on human operators. Systems and methods leverage probabilistic models of system and user behaviors built from logged data. A decision-theoretic analysis and corresponding models of ideal decisions about the transfer of calls from an automated system to a human operator are provided. The methods have application to a spectrum of call-handling systems including touch-tone and speech-recognition-based systems.
Description
- The present invention relates generally to systems and methods that facilitate communications between devices, systems, processes, and/or individuals. More particularly, the present invention relates to employment of dynamic policies for transferring people from an automated call routing system to a human operator given the likelihood of success, the effort associated with continuing to interact with an automated system and the wait times associated with entering a queue for a human operator.
- Many users have had experiences with the use of automated speech-recognition based call routing systems for accessing data and services by telephone. Such systems have proliferated lately with the advent of more accurate spoken command-and-control and continuous speech recognition systems for limited domains. Even when the scope of an application is restricted, however, such systems can produce frustrating failures. To address user frustration, many designers put in place options for callers to be immediately routed to a live, human operator. Beyond speech recognition systems, other automated systems have been deployed such as the use of touch-tone driven menu systems. Such systems may frustrate users in providing a fixed set of options that either may not appear to offer the right options, offer options that have understandable relevance, and/or may appear to route the caller to a frustrating sequence of options rather than converging quickly to a desired setting.
- A growing number of organizations are using automated call-routing systems that employ touch-tone routing or speech recognition and natural language processing to assist users with navigation. Automated methods are employed to reduce the load on operators or to reduce costs associated with staffing of call reception and routing functions. However, failures and frustrations of automated reception and routing can be costly to callers and to organizations employing them as callers may have to put in more effort, wait longer times, or simply hang up in frustration and take their business elsewhere.
- There has been a recent growth in the popularity of deploying speech recognition systems. One reason for the popularity of such systems is their promise at providing a more natural interface than touch-tone routing. However, organizations have been discovering that voice routing still falls short of the quality of service that a live, human operator can provide. In one example, an automated call routing system may field all internal directory assistance calls. Using speech recognition, the system attempts to uniquely identify one of over thousands of possible name entries in a company's global address book. Needless to say, a vocabulary of this size is bound to have deleterious effects on the performance of the speech engine.
- The following presents a simplified summary of the invention in order to provide a basic understanding of some aspects of the invention. This summary is not an extensive overview of the invention. It is not intended to identify key/critical elements of the invention or to delineate the scope of the invention. Its sole purpose is to present some concepts of the invention in a simplified form as a prelude to the more detailed description that is presented later.
- The present invention relates to guiding calls in accordance with an automated call routing system. One or more decision models are provided that output policies for switching people from an ongoing automated system to a human operator based on context-sensitive analysis of the spoken dialog situation at hand. This also relates to applying decision-theoretic principles to reason about the ideal melding of people and automated reasoning systems. Thus, callers' experiences with a call routing system are optimized by developing models that consider the probability that callers will be ultimately successful (or not) in their collaboration with the automated system, the expected number of steps required for that success, and the current load on human personnel. In one aspect, a voice routing system for directory assistance can be analyzed via data logs of a system's performance. One or more probabilistic models can then be constructed from the data logs regarding the likelihood of success or failure with the automated system, and the number of steps, overall duration, effort, or frustration with the use of the automated system, given that there will be eventual success of the interaction. The present invention couples path-dependent probabilities from learned models with a decision analysis to optimally guide the transfer of calls to a human operator, given information about the expected current wait times associated with a transfer to a human operator.
- To the accomplishment of the foregoing and related ends, certain illustrative aspects of the invention are described herein in connection with the following description and the annexed drawings. These aspects are indicative of various ways in which the invention may be practiced, all of which are intended to be covered by the present invention. Other advantages and novel features of the invention may become apparent from the following detailed description of the invention when considered in conjunction with the drawings.
- FIG. 1 is a schematic block diagram illustrating an automated call routing and decision system in accordance with an aspect of the present invention.
- FIG. 2 is a diagram illustrating call routing statistical data in accordance with an aspect of the present invention.
- FIG. 3 is a diagram illustrating dialog features in accordance with an aspect of the present invention.
- FIG. 4 is diagram illustrating a probability tree in accordance with an aspect of the present invention.
- FIG. 5 is a diagram illustrating a dependency network for dialog features in accordance with an aspect of the present invention.
- FIG. 6 is a diagram illustrating a Markov Dependency network in accordance with an aspect of the present invention.
- FIG. 7 is a prior distribution of an expected number of dialog steps in accordance with an aspect of the present invention.
- FIG. 8 is a flow diagram illustrating systematic automated call routing according to an aspect of the present invention.
- FIG. 9 is a schematic block diagram illustrating a suitable operating environment in accordance with an aspect of the present invention.
- FIG. 10 is a schematic block diagram of a sample-computing environment with which the present invention can interact.
- The subject invention employs decision theory in connection with managing calls. More particularly, management of call traffic and handling of specific calls is important to many businesses, and the subject invention employs decision theory in connection with optimizing handling of calls. An automatic call answering system can be employed until the cost associated therewith (e.g., customer hanging up) outweighs the benefits (e.g., minimizing human intervention to deal with the call). Thus, a system in accordance with the invention employs a decision-theoretic based framework to take a most appropriate action (e.g., use automatic call answering system, end use of call answering system, switch to human operator, route call to different service . . . ) based upon utilities associated with a particular component for answering a call and/or handling of call traffic, state of caller(s), state of entity receiving call(s), bandwidth, resources, and nature of call, for example. Thus, the subject invention optimizes utilization of resources via employment of decision theory and one more models that are trained from past activity logs.
- As used in this application, the terms “component,” “model,” “system,” and the like are intended to refer to a computer-related entity, either hardware, a combination of hardware and software, software, or software in execution. For example, a component may be, but is not limited to being, a process running on a processor, a processor, an object, an executable, a thread of execution, a program, and/or a computer. By way of illustration, both an application running on a server and the server can be a component. One or more components may reside within a process and/or thread of execution and a component may be localized on one computer and/or distributed between two or more computers.
- As used herein, the term “inference” refers generally to the process of reasoning about or inferring states of the system, environment, and/or user from a set of observations as captured via events and/or data. Inference can be employed to identify a specific context or action, or can generate a probability distribution over states, for example. The inference can be probabilistic; that is, the computation of a probability distribution over states of interest based on a consideration of data and events. Inference can also refer to logical inferences, including deterministic techniques employed for composing higher-level events from a set of events and/or data. Such inference results in the construction of new events or actions from a set of observed events and/or stored event data, whether or not the events are correlated in close temporal proximity, and whether the events and data come from one or several event and data sources.
- Referring initially to FIG. 1, a call routing and
decision system 100 is illustrated in accordance with an aspect of the present invention. Thesystem 100 includes an automatedcall routing system 110 that is routinely employed for providingautomated responses 120 to one ormore callers 130. These systems include processing components, switching components, electronic directories, and associated software such as speech recognition components for communicating with thecallers 130 and routing calls to an identified individual at 140 (e.g., voice command indicating individuals last name). Ifcallers 130 have trouble making a connection with an individual or party, thecall routing system 110 can connect to ahuman operator 144 to provide further assistance to the callers. - In accordance with one aspect of the present invention, one or more decision models150 are employed with the
call routing system 110 to facilitate efficient operations of thesystem 100, provide more efficient coupling between callers and respondents, and mitigate caller frustration when interacting with such systems. In one aspect, this achieved by training the decision models 150 via adata log 160 that has recorded data of past activities and interactions with thecall routing system 110. Output from the decision models 150 is then employed for call routing determinations. Such data includes statistical information such as how often speakers have been found or not found, how often an operator has been requested and so forth wherein this data is described in more detail below with respect to FIG. 2. After the decision models have been trained, thecall routing system 100 works in concert with the models to facilitate call routing between callers and individuals to be contacted. As will be described in more detail below, the models 150 can be employed in accordance with dynamic policies relating to the costs and benefits of switching a caller to the human operator, for example. - Referring now to FIG. 2, a diagram200 is illustrates statistical data that is analyzed in accordance with an aspect of the present invention. Although various categories of data are depicted by the diagram 200, it is to be appreciated that other categories of statistics relating to call routing performance can be similarly analyzed (e.g., average amount of time to connect). To assess the overall performance of the system described above with respect to FIG. 1, over 250 megabytes of data logs covering a period of roughly one year was obtained. The log contained close to 60,000 transcriptions of individual sessions with a call routing system, capturing respective system and caller actions. Some possible outcomes for a given session are as follows:
- SpeakFound: The system finds the name in the directory.
- SpeakNotFound: The system infers that the person requested may not be in the directory, and routes the call to an operator.
- OperatorRequest: The user requests an operator by pressing ‘0’.
- HelpRequest: The user tries to request help by pressing ‘*’ or ‘#’.
- HangUp: The user quits during the session.
- MaxErrrors: The system reaches the maximum number of allowed mistakes, and routes the call to an operator.
- NotReady: The system is temporarily out of service.
- Undefined: The user tries to press a numeric key.
- The diagram200 depicts an overall breakdown of all individual sessions into their possible outcomes. Due primarily to the large list of employee names in a sample company directory, a call routing system without associated models and policies achieved a dismal success rate of 45%, corresponding to the percentage of SpeakFound outcomes where the system correctly identified the proper name without the user attempting to request help from a human operator. The success rate jumps to 66% when session logs in which the caller did not even attempt to speak a name are removed. The situations where no interaction is attempted disclose the unsettling fact that many callers are likely to completely bypassing interaction with the system. In fact, these “no-name” attempts comprised 31% of the entire data, 85% of all OperatorRequest outcomes and 53% of all HangUp outcomes. In other words, roughly one out of every three callers decide to completely avoid interacting with the system unaided by attendant models. A longitudinal analysis of no-name cases over several months revealed that users appeared to be learning to immediately request a human operator. In fact, the correlation between a no-name situation and an “OperatorRequest” outcome was significant at ρ=0.88. This correlation went up to ρ=0.90 after the engineer responsible for the system added the phrase “for an operator, press ‘0’” to the introductory prompt, which had the effect of instructing users how to avoid working with the automated system. These findings were employed to characterize and optimize users' experiences via the models described above to more effectively enable callers to interact with the automated call routing system.
- Turning to FIG. 3, one or more dialog features300 are illustrated in accordance with an aspect of the present invention. Call routing systems typically treat policies for transferring users to an operator in an ad-hoc manner using handcrafted rules composed of various dialog features, such as the number of questions asked so far in a session. Rather than relying on intuition to construct the rules, the present invention builds probabilistic models from a large database of session logs with the intention of discovering dialog features that were predictive of success or failure. The dialog features employed for learning the models fall into four broad categories:
- System and user actions at310 (e.g., asking the user to pick between the top two guesses, the user has pressed a key, etc.)
- Session summary features at320 (e.g., number of name attempts detected, overall duration, etc.)
- n-best recognitions features at330 (e.g., range of confidence scores assigned by the speech recognition subsystem, mode, greatest consecutive score difference, count of the most frequent first/last/full names, etc.)
- Generalized temporal features340 (e.g., number of recurring first names that match in consecutive turns, etc.)
- The n-best recognitions features330 can be derived from a speech recognizer, and the generalized
temporal features 340 were included to cover trends between n-best lists. Using varying amounts of feature information, three classes of models (can be more or less than three) were built to estimate the likelihood that a session with the call routing system would eventually end in success or failure. - Referring now to FIG. 4, a
probability tree 400 showing the likelihood of success given any sequence of system actions is illustrated in accordance with an aspect of the present invention. Conditioning on a sequence of actions taken by a call routing system system, a likelihood of success can be determined, that is, p(SpeakFound|E), wherein observational evidence E refers to all system actions taken so far, by counting the number of cases along the action sequence that resulted in success over the total number of cases along the sequence. The probabilities can be visually displayed as atree 400 where each branch represents the system action, as shown in FIG. 4. - Table 1 below provides an expanded view of several dialog branches extracted from the
tree 400, where n refers to the number of callers reaching that state in the data logs. According to theprobability tree 400, the first system action [Operator Intro], which gives a standard prompt but appends the phrase “For an operator, please press ‘0’,” has only a 45% chance of succeeding. Interestingly, if the system subsequently asks the caller to repeat the entire name and then asks the caller to pick among three possible guesses, the likelihood of success increases to 66%. The increased probability of success for this initial interaction is atypical for dialog with the automated system as a whole. Typically, the longer the sequence of actions or path along the tree, the less likely it is that success will be achieved.TABLE 1 Expanded portion of several branches of the tree capturing probabilities of success given path. [Operator Intro] (p: 0.45, n: 51134) [Operator Intro] >> [Repeat full name] (p: 0.45, n: 20570) [Operator Intro] >> [Repeat full name] >> [Confirm from 3] (p: 0.66, n: 156) [Operator Intro] >> [Repeat full name] >> [Confirm from 3] >> [Repeat first name] (p: 0.17, n: 24) [Operator Intro] >> [Repeat full name] >> [Confirm from 3] >> [Repeat first name] >> [Repeat first name] (p: 0, n: 8) - The advantage of the marginal
probability tree model 400 is simplicity. The data that is required to build thetree 400 is a sequence of system actions for respective sessions and the ultimate outcome. As we will be described in the following section, the sequence of system actions happens to be the most predictive factor in determining success. The drawback to thismodel 400 is that the longer the sequence of actions, or path down the tree, the sparser the set of cases for building robust predictors of the likelihood of success. - FIG. 5 illustrates a
dependency network 500 for dialog features in accordance with an aspect of the present invention. Exploiting the four categories of dialog features discussed previously, another aspect of the present invention employs a Bayesian structure learning to build probabilistic models for predicting session outcome. A Bayesian learning tool that performs structure search and model scoring for different predictive models given a data set can be employed to learn a predictive model. As an example, the WinMine toolkit (Chickering et al. 1997) can be employed to build a dependency network and associated decision tree considering session outcome and other dialog features as input variables. - The top five dialog features for predicting outcome are displayed in the diagram500 in order of dependency strength are as follows: (1) the sequence of system actions, (2) the count or number of alternates in the n-best recognitions list, (3) the number of times the user attempted to speak a name, (4) the largest confidence score assigned by the system, and (5) the number of dialog turns—defined as a question-answer pair.
- The
dependency model 500 resonates with findings in the probability tree in that the dependency with the strongest link to outcome is the sequence of system actions. In the corresponding decision tree for the dependency network, it was found that the first decision split occurs when the system either does or does not identify the proper name after one confirmation attempt. If the confirmation is successful, then the likelihood of a SpeakFound session is almost certain at 99%. Otherwise, thedecision tree 500 considers other dialog features to predict success. Consistent with the idea that the longer the sequence of actions, the lower the chance of success, it was found that two of the next five strongest connections were related to the length of the dialog. The remaining three features were parameters output by the speech recognizer. - Given that the likelihood of success is closely tied to the length of the dialog, analysis was directed at how the decision rule split the fifth strongest dependency, the number of dialog turns. Since the split occurred at 2 dialog turns, the data set was divided by the number of turns and models were constructed for the first turn, second turn, and greater than two turns.
TABLE 2 Top five dependencies for one dialog turn, two turns, and more than two turns. Model Top Second Third Fourth Fifth 1 turn Number Range of Skewness Number Kurtosis of scores of scores of dialog of scores alternates turns 2 Number Redundant Number Redundant Count of turns of dialog first of last names n-Best turns names alternates Lists >2 Number Number Session Sum of Number turns of names of dialog duration scores of heard turns alternates - Table 2 displays the top five influencing variables for the respective dependency network models. Notice that for the first turn, almost all of the top five variables relate to the n-best recognition list generated by the speech recognizer, including the distribution characteristics of the confidence scores such as skewness and kurtosis. In moving to the second turn, however, generalized temporal features come into play, such as the maximum number of times a first or last name from the first dialog turn shows up again in the second turn. After two turns, the dialog features relate mostly to the length of the dialog.
- Referring to FIG. 6, a
Markov Dependency network 600 in accordance with an aspect of the present invention. In order to flesh out features from the speech recognizer that might be predictive of performance in the next turn, temporal dependency networks for Markov pairs of n-best recognition list features were constructed. FIG. 6 displays theMarkov dependency network 600. Since the count or number of alternates in the n-best recognition list was consistently selected to be one of the most strongest dependencies, the dependency network shows how it is possible to predict the number of alternates in turn t from n-best list features in turn t−1. - In order to evaluate the predictive performance of the probabilistic models described above, the classification accuracy of the learned models was compared against a marginal model capturing the overall run-time statistics for the training set. The models resulted from splitting the original dataset 70/30 into training and holdout data sets. As the probabilistic models were targeted for guiding decisions about dispatch, the outcome variable was circumscribed into two possible classification states: success versus failure, where success corresponds to the SpeakFound state. Table 3 below presents the results of the classification task on the holdout data for models examining the first dialog turn, the second turn, greater than two turns, and finally, the complete dataset.
- The partial-dialog models outperform the full model as well as the marginal for respective dialog turn datasets with the highest accuracy on the first dialog turn at 94% accuracy. The full model also achieves 85% on the complete dataset but provides poorer results when the dataset is decomposed by dialog turn. It is understood that increases in the amount of data can boost the partial models for dialog turn over the marginal models.
TABLE 3 Classification accuracy of the marginal, full, and partial-dialog models. Turn 1Turn 2Turn > 2 Complete Model (5792) (3741) (2652) (12185) Marginal 0.6979 0.6867 0.5199 0.6516 Full 0.7029 0.699 0.3872 0.8514 Turn 10.9404 n/a n/a n/a Turn 2 n/a 0.8083 n/a n/a Turn > 2 n/a n/a 0.7164 n/a - Beyond probing the probabilistic relationships among dialog behavior and success, a goal of the present invention has been to harness the probabilistic models in dynamic decisions about the costs and benefits of shifting a caller to a human operator. Having access to probabilities of the eventual success of a session with an automated system, as a function of the observed state of a dialog, provides a control surface enabling such decision-making.
- Qualitatively, probabilistic models with the ability to provide predictions about outcomes provide an immediate way for administrators of automated call routing systems to specify preferences regarding the transfer of callers to a human operator. Such preferences can be represented as a tolerated threshold on failure as a function of the current expected time that callers will have to wait for a human operator, given the current load on operators. The probabilistic models can also be employed in call center design. Staffing decisions can be made with the overall system model, constructed by taking into consideration the probabilistic performance of an automated system to route calls successfully, preferences about wait time, characterization of caller volumes, and the time required for addressing callers in a queue waiting for an operator.
- Systems and methods can be provided for handling the overall challenge of optimizing a call routing system design, based on a queue-theoretic formulation. In this invention, a decision-theoretic analysis is provided that minimizes the expected wait time for a caller, given observations about the nature and progress of dialog with an automated system.
- Assuming that the utility for a user, u(n,m,w), associated with the process of call routing, is a function of the number of dialog steps taken so far with the automated system, n, the total additional expected number of steps that will be taken for the current call with an automated routing system, m, and the wait time, w, associated with a transfer to a human operator should a transfer occur. It is noted that, in the general case, not only should the total time be considered, but the nature of the interaction steps. As an example, people may be extremely frustrated with the poor recognitions of the system, even if the overall outcome is accurate. Beyond the number of turns and wait times, the utility of an interaction for a caller may be influenced by other factors. For example, callers may have a negative emotional reaction to working with an automated system versus a human operator. Such factors can be folded into a cost-benefit analysis of routing actions under uncertainty, considering the number and nature of each step in a dialog.
- FIG. 7 is a diagram700 illustrating prior distribution of the expected number of dialog steps conditioned on the success of the automated call routing system. To continue the analysis from the discussion relating to FIG. 6, it can be assumed that the utility of an interaction is captured by the time cost of the interaction. The analysis can be generalized with a conversion of steps to an effective total time of an interaction, where frustration is captured by increases in the effective total time of specific steps. Thus, in one class of solutions, an assumption can be made that u(n,m,w) is captured by a measure of effective time, t(n,m,w)=t(n)+t(m)+w.
- The expected number of additional steps can be computed at each point in a dialog for the case where there is eventual success with the use of the automated system, and the case where the automated routing fails for any reason (e.g., the user presses the “O” key to access a human operator) and the user is immediately routed to a human operator. FIG. 7 displays the expected number of steps for success at 0 steps—the outset of an interaction.
- The present invention employs p(xfer|E,ξ) to refer to the probability that the interaction will eventually fail at some point and the user will be immediately routed to a human operator at that point. At each point in an interaction, pre-compute and make available, p(xfer|E,ξ) and p (success|E,ξ)=1−p(xfer|E,ξ). Also pre-compute the probability distributions, p(m|E,xfer,ξ)—the probability distribution over the additional number of steps given an eventual failure, p(m|E,success,ξ)—the probability distribution over the number of additional steps given a successful interaction with the automated system, and the expected number of steps for each of these conditions, labeled <m> and <m′>, respectively.
- The expected additional wait time with continuing the automated interaction, ta at each point in a dialog under uncertainty in failure is as follows:
- t a =p(xfer|E,ξ) (t(<m>)+w)+(1−p(xfer|E,ξ)) t(<m′>)
Equation 1 - The wait time associated with an immediate transfer into the queue for interacting with a human operator is w. This time varies and the current value can be measured or estimated at any moment by measuring the average recent wait times, or by checking the queue of callers waiting for an operator. At the current position in a dialog, the expected wait time associated with continuing the automated interaction versus making an immediate automated transfer to the waiting queue for a human operator can be compared. That is, continue to determine if ta>w. If the expected wait time is greater for continuing to engage the user with the automated system, then transfer the user to the queue for a human operator.
- Beyond taking a myopic analysis, the present invention can perform different amounts of look ahead, and invoke
Equation 1 to compute the expected wait times at points further downstream in the dialog, folding in a consideration of the uncertainty that the user will take different paths conditioned on the current path, and will successfully reach each of the downstream points. Employing the decision-theoretic measure promises to minimize the total wait time for users. - Also, the cost of handling a call with a human operator, C, for an organization and take an organizational perspective on decision making can be considered. Rather than just modeling the cost to a user in terms of time and frustration, the following, considers an organization's assignment of the utility as a function the cost of the handling of a call by a human operator, in addition to the organizations assessment of the cost associated with the user's time and frustration,
- Utility of call handling=p(xfer|E,ξ) u(t(n)+t(<m>)+w), C)+(1−p(xfer|E,ξ)) u(t(n)+t(<m′>).
Equation 2 - In this aspect, consider an organization's assessment of a utility, u(t, C), capturing the overall cost as a function of the wait time that a caller experiences and the monetary cost of handling a call. Note that the cost may not be constant; evidence about a user's selections and responses may be used to update a probability distribution over the cost p(C|E,ξ), and the expected costs can be substituted in
Equation 2. - As another extension, consider also the cost of a total failure of the communication, based on folding in the likelihood that a user will simply hang up in frustration at each point in the interaction, as a function of the path they are on and the wait times in a queue. With this extension, the overall organization's consideration of the expected cost of handling a call is,
- Utility of call handling=1−p(fail|E,ξ) (p(xfer|E,ξ) u(t(n)+t(<m>)+w, C, success)+(1−p(xfer|E,ξ)(u(t(n)+t(<m′>),0, success)+p(fail|E,ξ) u(u(t(n)+t(<m>),0, fail)
Equation 3 - In this aspect, consider the organization's assessment of a utility, u(t, C, outcome), of the cost as a function of the wait time that a caller experiences, the monetary cost of handling a call, and whether the call eventually is completed or fails with a disconnection. The loss of a caller is denoted as p(fail|E,ξ). Such a utility model can be expanded to include more detailed modeling about whether a caller is lost to an organization forever or until another attempt is made.
- FIG. 8 illustrates a methodology for call routing and decision making in accordance the present invention. While, for purposes of simplicity of explanation, the methodology is shown and described as a series of acts, it is to be understood and appreciated that the present invention is not limited by the order of acts, as some acts may, in accordance with the present invention, occur in different orders and/or concurrently with other acts from that shown and described herein. For example, those skilled in the art will understand and appreciate that a methodology could alternatively be represented as a series of interrelated states or events, such as in a state diagram. Moreover, not all illustrated acts may be required to implement a methodology in accordance with the present invention.
- Proceeding to802, the present invention considers the various interactions and performance of a real-world speech-recognition based on an automated call routing system. As noted above, this can include an analysis of the various system components and operator interactions affecting one or more variables of system performance. At 804, data is gathered to determine various performance aspects of the system. This can include call answering statistics, routing statistics, success or failure statistics based on various factors such as the amount of time a caller has to wait before being directed into a queue to speak with a human operator, for example. At 808, one or more probabilistic models can be constructed from the data that is employed to provide probabilities of different paths through the call routing system, including such information as whether an interaction with the automated system will be successful. At 812, a decision-theoretic analysis of the value of switching to a human operator at different points in a user's interaction with the automated routing system is also provided.
- With reference to FIG. 9, an
exemplary environment 910 for implementing various aspects of the invention includes acomputer 912. Thecomputer 912 includes aprocessing unit 914, asystem memory 916, and asystem bus 918. Thesystem bus 918 couples system components including, but not limited to, thesystem memory 916 to theprocessing unit 914. Theprocessing unit 914 can be any of various available processors. Dual microprocessors and other multiprocessor architectures also can be employed as theprocessing unit 914. - The
system bus 918 can be any of several types of bus structure(s) including the memory bus or memory controller, a peripheral bus or external bus, and/or a local bus using any variety of available bus architectures including, but not limited to, 11-bit bus, Industrial Standard Architecture (ISA), Micro-Channel Architecture (MSA), Extended ISA (EISA), Intelligent Drive Electronics (IDE), VESA Local Bus (VLB), Peripheral Component Interconnect (PCI), Universal Serial Bus (USB), Advanced Graphics Port (AGP), Personal Computer Memory Card International Association bus (PCMCIA), and Small Computer Systems Interface (SCSI). - The
system memory 916 includesvolatile memory 920 andnonvolatile memory 922. The basic input/output system (BIOS), containing the basic routines to transfer information between elements within thecomputer 912, such as during start-up, is stored innonvolatile memory 922. By way of illustration, and not limitation,nonvolatile memory 922 can include read only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable ROM (EEPROM), or flash memory.Volatile memory 920 includes random access memory (RAM), which acts as external cache memory. By way of illustration and not limitation, RAM is available in many forms such as synchronous RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), Synchlink DRAM (SLDRAM), and direct Rambus RAM (DRRAM). -
Computer 912 also includes removable/non-removable, volatile/non-volatile computer storage media. FIG. 9 illustrates, for example adisk storage 924.Disk storage 924 includes, but is not limited to, devices like a magnetic disk drive, floppy disk drive, tape drive, Jaz drive, Zip drive, LS-100 drive, flash memory card, or memory stick. In addition,disk storage 924 can include storage media separately or in combination with other storage media including, but not limited to, an optical disk drive such as a compact disk ROM device (CD-ROM), CD recordable drive (CD-R Drive), CD rewritable drive (CD-RW Drive) or a digital versatile disk ROM drive (DVD-ROM). To facilitate connection of thedisk storage devices 924 to thesystem bus 918, a removable or non-removable interface is typically used such asinterface 926. - It is to be appreciated that FIG. 9 describes software that acts as an intermediary between users and the basic computer resources described in
suitable operating environment 910. Such software includes anoperating system 928.Operating system 928, which can be stored ondisk storage 924, acts to control and allocate resources of thecomputer system 912.System applications 930 take advantage of the management of resources byoperating system 928 throughprogram modules 932 andprogram data 934 stored either insystem memory 916 or ondisk storage 924. It is to be appreciated that the present invention can be implemented with various operating systems or combinations of operating systems. - A user enters commands or information into the
computer 912 through input device(s) 936.Input devices 936 include, but are not limited to, a pointing device such as a mouse, trackball, stylus, touch pad, keyboard, microphone, joystick, game pad, satellite dish, scanner, TV tuner card, digital camera, digital video camera, web camera, and the like. These and other input devices connect to theprocessing unit 914 through thesystem bus 918 via interface port(s) 938. Interface port(s) 938 include, for example, a serial port, a parallel port, a game port, and a universal serial bus (USB). Output device(s) 940 use some of the same type of ports as input device(s) 936. Thus, for example, a USB port may be used to provide input tocomputer 912, and to output information fromcomputer 912 to anoutput device 940.Output adapter 942 is provided to illustrate that there are someoutput devices 940 like monitors, speakers, and printers, amongother output devices 940, that require special adapters. Theoutput adapters 942 include, by way of illustration and not limitation, video and sound cards that provide a means of connection between theoutput device 940 and thesystem bus 918. It should be noted that other devices and/or systems of devices provide both input and output capabilities such as remote computer(s) 944. -
Computer 912 can operate in a networked environment using logical connections to one or more remote computers, such as remote computer(s) 944. The remote computer(s) 944 can be a personal computer, a server, a router, a network PC, a workstation, a microprocessor based appliance, a peer device or other common network node and the like, and typically includes many or all of the elements described relative tocomputer 912. For purposes of brevity, only amemory storage device 946 is illustrated with remote computer(s) 944. Remote computer(s) 944 is logically connected tocomputer 912 through anetwork interface 948 and then physically connected viacommunication connection 950.Network interface 948 encompasses communication networks such as local-area networks (LAN) and wide-area networks (WAN). LAN technologies include Fiber Distributed Data Interface (FDDI), Copper Distributed Data Interface (CDDI), Ethernet/IEEE 1102.3, Token Ring/IEEE 1102.5 and the like. WAN technologies include, but are not limited to, point-to-point links, circuit switching networks like Integrated Services Digital Networks (ISDN) and variations thereon, packet switching networks, and Digital Subscriber Lines (DSL). - Communication connection(s)950 refers to the hardware/software employed to connect the
network interface 948 to thebus 918. Whilecommunication connection 950 is shown for illustrative clarity insidecomputer 912, it can also be external tocomputer 912. The hardware/software necessary for connection to thenetwork interface 948 includes, for exemplary purposes only, internal and external technologies such as, modems including regular telephone grade modems, cable modems and DSL modems, ISDN adapters, and Ethernet cards. - FIG. 10 is a schematic block diagram of a sample-
computing environment 1000 with which the present invention can interact. Thesystem 1000 includes one or more client(s) 1010. The client(s) 1010 can be hardware and/or software (e.g., threads, processes, computing devices). Thesystem 1000 also includes one or more server(s) 1030. The server(s) 1030 can also be hardware and/or software (e.g., threads, processes, computing devices). Theservers 1030 can house threads to perform transformations by employing the present invention, for example. One possible communication between aclient 1010 and aserver 1030 may be in the form of a data packet adapted to be transmitted between two or more computer processes. Thesystem 1000 includes acommunication framework 1050 that can be employed to facilitate communications between the client(s) 1010 and the server(s) 1030. The client(s) 1010 are operably connected to one or more client data store(s) 1060 that can be employed to store information local to the client(s) 1010. Similarly, the server(s) 1030 are operably connected to one or more server data store(s) 1040 that can be employed to store information local to theservers 1030. - What has been described above includes examples of the present invention. It is, of course, not possible to describe every conceivable combination of components or methodologies for purposes of describing the present invention, but one of ordinary skill in the art may recognize that many further combinations and permutations of the present invention are possible. Accordingly, the present invention is intended to embrace all such alterations, modifications and variations that fall within the spirit and scope of the appended claims. Furthermore, to the extent that the term “includes” is used in either the detailed description or the claims, such term is intended to be inclusive in a manner similar to the term “comprising” as “comprising” is interpreted when employed as a transitional word in a claim.
Claims (36)
1. An automated call routing system, comprising:
an automated call routing component to route incoming calls to members of an organization and providing automated responses to one or more callers; and
a decision model associated with the automated call routing component to mitigate transferring the calls to an operator.
2. The system of claim 1 , further comprising a speech recognition component for communicating with the callers.
3. The system of claim 1 , the decision model is trained from a data log that has recorded data of past activities and interactions with the call routing component.
4. The system of claim 3 , the data log contains data relating to at least one of a Speaker Found, a Speaker Not Found, an OperatorRequest, a Help Request, a Hang Up, a Maximum number of Errrors, a Not Ready indication, and an Undefined category.
5. The system of claim 1 , the decision model processes one or more dialog features including at least one of system and user actions, session summary features, n-best recognitions features, and generalized temporal features.
6. The system of claim 5 , the n-best recognitions features are derived from a speech recognizer, and the generalized temporal features are included to cover trends between one or more n-best lists.
7. The system of claim 1 , the decision model employs a probability tree determining a likelihood of success given a sequence of system actions.
8. The system of claim 7 , the decision model determining p(SpeakFound|E), wherein observational evidence E refers to system actions taken, by counting a number of logged cases along an action sequence that resulted in success over a total number of cases along the sequence, wherein p is a probability.
9. The system of claim 1 , decision model employs a dependency network that processes one or more categories of dialog features as input variables.
10. The system of claim 9 , the decision model processes at least one of a sequence of system actions, a count or number of alternates in an n-best recognitions list, a number of times a user attempted to speak a name, a largest score assigned by a call routing system, and a number of dialog turns—defined as a question-answer pair.
12. The system of claim 1 , the decision model employs a Markov Dependency network.
13. The system of claim 12 , further comprising a component to increase an amount of data in order to boost a partial model for dialog turns over a marginal model.
14. The system of claim 1 , the decision model includes probabilistic models to perform dynamic decisions about costs and benefits of shifting a caller to a human operator.
15. The system of claim 14 , the probabilistic models provide predictions about outcomes to enable administrators of automated call routing systems to specify preferences regarding the transfer of callers to a human operator.
16. The system of claim 15 , the preferences are represented as a tolerated threshold on failure as a function of a current expected time that callers have to wait for a human operator, given a current load on operators. The probabilistic models can also be employed in call center design.
17. The system of claim 1 , the decision model is employed to facilitate staffing decisions by taking into consideration at least one of probabilistic performance of an automated system to route calls successfully, preferences about wait time, characterization of caller volumes, and time required for addressing callers in a queue waiting for an operator.
18. The system of claim 17 , the queue is optimized based on a queue-theoretic formulation.
19. A computer readable medium having computer readable instructions stored thereon for implementing at least one of the call routing component and the decision model of claim 1 .
20. A system that facilitates call routing, comprising:
means for interacting with a caller;
means for automatically directing the caller to a user; and
means for performing a decision theoretic analysis before directing the caller to a user, the decision-theoretic includes a cost benefit analysis weighing the benefits of transferring the caller to an operator.
21. A method for automatically routing calls, comprising:
determining a utility model for employment with a call routing system;
training the utility model from a log of past system call activities; and
automatically directing calls to at least one of an organization member and an operator.
22. The method of claim 21 , the utility model is applied to a user function, u(n,m,w), associated with a process of call routing, the user function is a function of a number of automated dialog steps taken, n, a total expected number of steps that will be taken with an automated routing system, m, and a wait time, w, for transferring to a human operator.
23. The method of claim 22 , further comprising processing user frustrations.
24. The method of claim 22 , further comprising processing negative emotional reactions to working with an automated system versus a human operator.
25. The method of claim 22 , further comprising performing a cost-benefit analysis of routing actions under uncertainty, considering a number and nature of at least one step in a dialog.
26. The method of claim 21 , further comprising determining a utility of an interaction in accordance with a time cost of an interaction.
27. The method of claim 26 , further comprising generalizing a conversion of steps to an effective total time of an interaction, wherein frustration is captured by increases in an effective total time of specific steps.
28. The method of claim 26 , further comprising a pre-computation that is performed to yield, p(xfer|E,ξ) and p(success|E,ξ)=1−p(xfer|E,ξ).
29. The method of claim 26 , further comprising a pre-computation of probability distributions, p(m|E,xfer,ξ) and p(m|E,success,ξ) and an expected number of steps for conditions, labeled <m> and <m′>, respectively.
30. The method of claim 21 , further comprising determining an expected total wait time with continuing an automated interaction, ta at respective points in a dialog under uncertainty in failure as:
ta =p(xfer|E,ξ) (t(<m>)+w)+(1−p(xfer|E,ξ)) (t(<m′>)
wherein a wait time associated with a courteous immediate transfer into a queue for interacting with a human operator is w.
31. The method of claim 30 , further comprising determining a utility of call handling as follows:
Utility of call handling=p(xfer|E,ξ) u(t(n)+t(<m>)+w), C)+(1−p(xfer|E,ξ)) u(t(n)+t(<m′>)
32. The method of claim 31 , further comprising determining the utility of call handling as follows:
Utility of call handling=1−p(fail|E,ξ) (p(xfer|E,ξ) u(t(n)+t(<m>)+w, C, success)+(1−p(xfer|E,ξ)(u(t(n)+t(<m′>),0, success)+p(fail|E,ξ) u(u(t(n)+t(<m>),0, fail).
33. The method of claim 32 , where cost of handling the call with a human operator C depends on needs or goals of the caller, and is inferred from evidence.
34. The method of claim 32 , further comprising determining expected costs via inference of a probability distribution over Cost given evidence gathered so far, p(C|E,ξ).
35. The method of claim 21 , further comprising providing online sensing of current wait times for calls being transferred to a human operator.
36. The method of claim 21 , further comprising at least one of: creating an end-to-end system that continues to at least one of log, monitor, and build models; and
automatically setting parameters, generating reports, and generating traces, for validation and auditing of actions.
37. The method of claim 21 supporting an application including at least one of touch-tone routing and speech recognition.
Priority Applications (9)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US10/610,486 US20040264677A1 (en) | 2003-06-30 | 2003-06-30 | Ideal transfer of call handling from automated systems to human operators based on forecasts of automation efficacy and operator load |
US10/827,873 US7742591B2 (en) | 2003-06-30 | 2004-04-20 | Queue-theoretic models for ideal integration of automated call routing systems with human operators |
JP2004152123A JP2005027283A (en) | 2003-06-30 | 2004-05-21 | Ideal call processing transfer from automatic system to human operator, based on prediction of automation validity and prediction of operator load |
AT04102315T ATE420537T1 (en) | 2003-06-30 | 2004-05-26 | IDEAL TRANSFER OF CALL CONTROL FROM AUTOMATED SYSTEMS TO HUMAN OPERATORS |
DE602004018875T DE602004018875D1 (en) | 2003-06-30 | 2004-05-26 | Ideal assignment of call control from automated systems to human operators |
EP04102315A EP1494499B1 (en) | 2003-06-30 | 2004-05-26 | Ideal transfer of call handling from automated systems to human operators |
BR0403831-2A BRPI0403831A (en) | 2003-06-30 | 2004-05-27 | Optimal call handling transfer from automated systems to human operators based on automation effectiveness predictions and operator load |
CN2004100631081A CN1592427B (en) | 2003-06-30 | 2004-05-28 | Ideal transfer of call handling from automated systems to human operators |
KR1020040039095A KR101099231B1 (en) | 2003-06-30 | 2004-05-31 | Ideal transfer of call handling from automated systems to human operator based on forecasts of automation efficacy and operator load |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US10/610,486 US20040264677A1 (en) | 2003-06-30 | 2003-06-30 | Ideal transfer of call handling from automated systems to human operators based on forecasts of automation efficacy and operator load |
Related Child Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US10/827,873 Continuation-In-Part US7742591B2 (en) | 2003-06-30 | 2004-04-20 | Queue-theoretic models for ideal integration of automated call routing systems with human operators |
Publications (1)
Publication Number | Publication Date |
---|---|
US20040264677A1 true US20040264677A1 (en) | 2004-12-30 |
Family
ID=33435409
Family Applications (2)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US10/610,486 Abandoned US20040264677A1 (en) | 2003-06-30 | 2003-06-30 | Ideal transfer of call handling from automated systems to human operators based on forecasts of automation efficacy and operator load |
US10/827,873 Expired - Fee Related US7742591B2 (en) | 2003-06-30 | 2004-04-20 | Queue-theoretic models for ideal integration of automated call routing systems with human operators |
Family Applications After (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US10/827,873 Expired - Fee Related US7742591B2 (en) | 2003-06-30 | 2004-04-20 | Queue-theoretic models for ideal integration of automated call routing systems with human operators |
Country Status (8)
Country | Link |
---|---|
US (2) | US20040264677A1 (en) |
EP (1) | EP1494499B1 (en) |
JP (1) | JP2005027283A (en) |
KR (1) | KR101099231B1 (en) |
CN (1) | CN1592427B (en) |
AT (1) | ATE420537T1 (en) |
BR (1) | BRPI0403831A (en) |
DE (1) | DE602004018875D1 (en) |
Cited By (40)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20050055216A1 (en) * | 2003-09-04 | 2005-03-10 | Sbc Knowledge Ventures, L.P. | System and method for the automated collection of data for grammar creation |
US20050069122A1 (en) * | 2003-09-30 | 2005-03-31 | Xiaofan Lin | System and method for operator assisted automated call handling |
US20050286705A1 (en) * | 2004-06-16 | 2005-12-29 | Matsushita Electric Industrial Co., Ltd. | Intelligent call routing and call supervision method for call centers |
US20070025528A1 (en) * | 2005-07-07 | 2007-02-01 | Sbc Knowledge Ventures, L.P. | System and method for automated performance monitoring for a call servicing system |
US20070198272A1 (en) * | 2006-02-20 | 2007-08-23 | Masaru Horioka | Voice response system |
US20080077470A1 (en) * | 2006-09-26 | 2008-03-27 | Fujitsu Limited | Respondent-information output apparatus, respondent-information output method, and computer product |
US7436948B1 (en) * | 2004-12-23 | 2008-10-14 | Sprint Spectrum L.P. | Method and system for timed interaction with an interactive voice response |
US20090100047A1 (en) * | 2007-10-15 | 2009-04-16 | Chacha Search, Inc. | Method and system of managing and using profile information |
US7624007B2 (en) * | 1999-11-12 | 2009-11-24 | Phoenix Solutions, Inc. | System and method for natural language processing of sentence based queries |
US8204205B1 (en) * | 2007-02-08 | 2012-06-19 | At&T Intellectual Property Ii, L.P. | Method for increasing the efficiency of automated telephony systems |
US8280030B2 (en) | 2005-06-03 | 2012-10-02 | At&T Intellectual Property I, Lp | Call routing system and method of using the same |
US8300798B1 (en) * | 2006-04-03 | 2012-10-30 | Wai Wu | Intelligent communication routing system and method |
US8379830B1 (en) * | 2006-05-22 | 2013-02-19 | Convergys Customer Management Delaware Llc | System and method for automated customer service with contingent live interaction |
US8494152B1 (en) * | 2006-02-28 | 2013-07-23 | Allstate Insurance Company | Systems and methods for automated call-handling and processing |
US8600757B2 (en) * | 2005-01-07 | 2013-12-03 | At&T Intellectual Property Ii, L.P. | System and method of dynamically modifying a spoken dialog system to reduce hardware requirements |
US8601030B2 (en) * | 2011-09-09 | 2013-12-03 | International Business Machines Corporation | Method for a natural language question-answering system to complement decision-support in a real-time command center |
US20130329880A1 (en) * | 2012-06-11 | 2013-12-12 | Interactive Intelligence, Inc. | Method and system for improving the productivity of calling agents and call yield |
US8751232B2 (en) | 2004-08-12 | 2014-06-10 | At&T Intellectual Property I, L.P. | System and method for targeted tuning of a speech recognition system |
US8824659B2 (en) | 2005-01-10 | 2014-09-02 | At&T Intellectual Property I, L.P. | System and method for speech-enabled call routing |
US9112972B2 (en) | 2004-12-06 | 2015-08-18 | Interactions Llc | System and method for processing speech |
US9213695B2 (en) | 2012-02-06 | 2015-12-15 | Language Line Services, Inc. | Bridge from machine language interpretation to human language interpretation |
US9213693B2 (en) | 2012-04-03 | 2015-12-15 | Language Line Services, Inc. | Machine language interpretation assistance for human language interpretation |
WO2016019194A1 (en) * | 2014-08-01 | 2016-02-04 | Genesys Telecommunications Laboratories, Inc. | System and method for case-based routing for a contact center |
US20160127553A1 (en) * | 2014-10-31 | 2016-05-05 | Avaya Inc. | System and method for managing resources of an enterprise |
US9473632B2 (en) | 2014-08-27 | 2016-10-18 | Avaya Inc. | System and method to improve self-service experience and optimize contact center resources |
WO2016167424A1 (en) * | 2015-04-16 | 2016-10-20 | 주식회사 플런티코리아 | Answer recommendation device, and automatic sentence completion system and method |
US9736308B1 (en) | 2002-03-07 | 2017-08-15 | Wai Wu | Intelligent communication routing |
US9781270B2 (en) | 2014-08-01 | 2017-10-03 | Genesys Telecommunications Laboratories, Inc. | System and method for case-based routing for a contact |
US9848084B2 (en) | 2014-08-01 | 2017-12-19 | Genesys Telecommunications Laboratories, Inc. | Adaptable business objective routing for a contact center |
US9860391B1 (en) | 2003-03-07 | 2018-01-02 | Wai Wu | Method and system for matching entities in an auction |
US20180286386A1 (en) * | 2017-03-28 | 2018-10-04 | International Business Machines Corporation | Deep language and acoustic modeling convergence and cross training |
US10230846B2 (en) * | 2016-05-17 | 2019-03-12 | Facebook, Inc. | Systems and methods for interacting with answering systems |
US10515156B2 (en) * | 2012-08-31 | 2019-12-24 | Verint Americas Inc | Human-to-human conversation analysis |
US10567975B2 (en) | 2005-10-04 | 2020-02-18 | Hoffberg Family Trust 2 | Multifactorial optimization system and method |
US10718031B1 (en) | 2014-11-03 | 2020-07-21 | Wai Wu | Method and system for matching entities in an auction |
US10943273B2 (en) | 2003-02-05 | 2021-03-09 | The Hoffberg Family Trust 2004-1 | System and method for determining contingent relevance |
US11250215B2 (en) * | 2019-04-16 | 2022-02-15 | International Business Machines Corporation | Form-based transactional conversation system design |
US11316980B2 (en) | 2019-11-26 | 2022-04-26 | International Business Machines Corporation | Agent to bot transfer |
US11822888B2 (en) | 2018-10-05 | 2023-11-21 | Verint Americas Inc. | Identifying relational segments |
US11861316B2 (en) | 2018-05-02 | 2024-01-02 | Verint Americas Inc. | Detection of relational language in human-computer conversation |
Families Citing this family (35)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US7330895B1 (en) | 2001-03-15 | 2008-02-12 | Microsoft Corporation | Representation, decision models, and user interface for encoding managing preferences, and performing automated decision making about the timing and modalities of interpersonal communications |
US7870240B1 (en) | 2002-06-28 | 2011-01-11 | Microsoft Corporation | Metadata schema for interpersonal communications management systems |
US7555533B2 (en) * | 2003-10-15 | 2009-06-30 | Harman Becker Automotive Systems Gmbh | System for communicating information from a server via a mobile communication device |
DE602004010054T2 (en) * | 2004-01-19 | 2008-03-06 | Harman Becker Automotive Systems Gmbh | Actuation of a speech dialogue system |
DE602004014893D1 (en) | 2004-01-29 | 2008-08-21 | Daimler Ag | Multimodal data entry |
DE602004017955D1 (en) * | 2004-01-29 | 2009-01-08 | Daimler Ag | Method and system for voice dialogue interface |
EP1562180B1 (en) * | 2004-02-06 | 2015-04-01 | Nuance Communications, Inc. | Speech dialogue system and method for controlling an electronic device |
JP5394739B2 (en) * | 2005-08-09 | 2014-01-22 | モバイル・ヴォイス・コントロール・エルエルシー | Voice-controlled wireless communication device / system |
US20070043604A1 (en) * | 2005-08-22 | 2007-02-22 | Aspect Communications Corporation | Methods and systems to obtain a relative frequency distribution describing a distribution of counts |
US7818417B2 (en) * | 2006-01-10 | 2010-10-19 | International Business Machines Corporation | Method for predicting performance of distributed stream processing systems |
US20070168457A1 (en) * | 2006-01-18 | 2007-07-19 | International Business Machines Corporation | Apparatus and method for addressing computer-related problems |
US7599861B2 (en) | 2006-03-02 | 2009-10-06 | Convergys Customer Management Group, Inc. | System and method for closed loop decisionmaking in an automated care system |
JP2007272123A (en) * | 2006-03-31 | 2007-10-18 | Toyota Central Res & Dev Lab Inc | Voice operation system |
US7809663B1 (en) | 2006-05-22 | 2010-10-05 | Convergys Cmg Utah, Inc. | System and method for supporting the utilization of machine language |
WO2008132572A1 (en) * | 2007-04-25 | 2008-11-06 | Van Den Berg Barend Frederick | Telephone call processing system amd method |
US8126723B1 (en) | 2007-12-19 | 2012-02-28 | Convergys Cmg Utah, Inc. | System and method for improving tuning using caller provided satisfaction scores |
US8086596B2 (en) * | 2008-05-15 | 2011-12-27 | International Business Machines Corporation | Healthcheck of information technology infrastructure based on log data |
US20110153381A1 (en) * | 2009-12-18 | 2011-06-23 | Saryu Shah | Method and System for Smart Queuing of Test Requests |
US8396204B2 (en) * | 2010-10-09 | 2013-03-12 | Hewlett-Packard Development Company, L.P. | Call center resource allocation |
US8639508B2 (en) * | 2011-02-14 | 2014-01-28 | General Motors Llc | User-specific confidence thresholds for speech recognition |
CN104781774A (en) * | 2012-09-12 | 2015-07-15 | 格林伊登美国控股有限责任公司 | System and method for dynamic configuration of contact centers via templates |
US9912813B2 (en) | 2012-11-21 | 2018-03-06 | Genesys Telecommunications Laboratories, Inc. | Graphical user interface with contact center performance visualizer |
US9628623B2 (en) | 2012-11-21 | 2017-04-18 | Genesys Telecommunications Laboratories, Inc. | Graphical user interface for monitoring and visualizing contact center routing strategies |
US9912812B2 (en) | 2012-11-21 | 2018-03-06 | Genesys Telecommunications Laboratories, Inc. | Graphical user interface for configuring contact center routing strategies |
CN103118208B (en) * | 2013-01-29 | 2015-09-02 | 江苏电力信息技术有限公司 | A kind of telephone traffic peak surge calling optimization method based on dynamic decision tree |
JP6111802B2 (en) * | 2013-03-29 | 2017-04-12 | 富士通株式会社 | Spoken dialogue apparatus and dialogue control method |
EP2838249A4 (en) * | 2013-07-04 | 2015-08-19 | Huawei Tech Co Ltd | Method, apparatus and system for voice call processing |
US9906648B2 (en) | 2014-09-23 | 2018-02-27 | Interactive Intelligence Group, Inc. | Method and system for prediction of contact allocation, staff time distribution, and service performance metrics in a multi-skilled contact center operation environment |
CN105206269A (en) * | 2015-08-14 | 2015-12-30 | 百度在线网络技术(北京)有限公司 | Voice processing method and device |
US9811519B2 (en) * | 2015-09-24 | 2017-11-07 | Conduent Business Services, Llc | Generative discriminative approach for transactional dialog state tracking via collective matrix factorization |
US11392896B2 (en) * | 2017-06-02 | 2022-07-19 | Apple Inc. | Event extraction systems and methods |
US11316977B2 (en) * | 2017-10-27 | 2022-04-26 | Tata Consultancy Services Limited | System and method for call routing in voice-based call center |
US10776252B1 (en) | 2018-11-27 | 2020-09-15 | Walgreen Co. | Crowd-sourced automatic generation of user interface tests for enterprise-specific mobile applications |
US11769038B2 (en) | 2019-04-12 | 2023-09-26 | Optum, Inc. | Contextually optimizing routings for interactions |
CN112365099B (en) * | 2020-12-08 | 2024-03-19 | 南京大学 | Non-deterministic separation web server cluster telescoping method |
Citations (34)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US4747054A (en) * | 1984-11-28 | 1988-05-24 | Conoco Inc. | Method for non-linear signal matching |
US5291550A (en) * | 1990-12-26 | 1994-03-01 | At&T Bell Laboratories | Dynamic network call distributor |
US5493692A (en) * | 1993-12-03 | 1996-02-20 | Xerox Corporation | Selective delivery of electronic messages in a multiple computer system based on context and environment of a user |
US5519773A (en) * | 1994-06-07 | 1996-05-21 | Siemens Colm Communications Inc. | Call sharing for inbound and outbound call center agents |
US5530744A (en) * | 1994-09-20 | 1996-06-25 | At&T Corp. | Method and system for dynamic customized call routing |
US5544321A (en) * | 1993-12-03 | 1996-08-06 | Xerox Corporation | System for granting ownership of device by user based on requested level of ownership, present state of the device, and the context of the device |
US5561711A (en) * | 1994-03-09 | 1996-10-01 | Us West Technologies, Inc. | Predictive calling scheduling system and method |
US5812865A (en) * | 1993-12-03 | 1998-09-22 | Xerox Corporation | Specifying and establishing communication data paths between particular media devices in multiple media device computing systems based on context of a user or users |
US5848131A (en) * | 1993-02-22 | 1998-12-08 | Murex Securities, Ltd. | Automatic information and routing system for telephonic services |
US5896448A (en) * | 1996-12-17 | 1999-04-20 | Bellsouth Corporation | Method and apparatus for routing calls based on call success history |
US5995805A (en) * | 1997-10-17 | 1999-11-30 | Lockheed Martin Missiles & Space | Decision-theoretic satellite communications system |
US6115462A (en) * | 1998-01-09 | 2000-09-05 | Gte Laboratories Incorporated | Method and apparatus for efficient call routing |
US20010040591A1 (en) * | 1998-12-18 | 2001-11-15 | Abbott Kenneth H. | Thematic response to a computer user's context, such as by a wearable personal computer |
US20010040590A1 (en) * | 1998-12-18 | 2001-11-15 | Abbott Kenneth H. | Thematic response to a computer user's context, such as by a wearable personal computer |
US20010043232A1 (en) * | 1998-12-18 | 2001-11-22 | Abbott Kenneth H. | Thematic response to a computer user's context, such as by a wearable personal computer |
US20020032689A1 (en) * | 1999-12-15 | 2002-03-14 | Abbott Kenneth H. | Storing and recalling information to augment human memories |
US20020044152A1 (en) * | 2000-10-16 | 2002-04-18 | Abbott Kenneth H. | Dynamic integration of computer generated and real world images |
US20020052930A1 (en) * | 1998-12-18 | 2002-05-02 | Abbott Kenneth H. | Managing interactions between computer users' context models |
US20020054174A1 (en) * | 1998-12-18 | 2002-05-09 | Abbott Kenneth H. | Thematic response to a computer user's context, such as by a wearable personal computer |
US20020054130A1 (en) * | 2000-10-16 | 2002-05-09 | Abbott Kenneth H. | Dynamically displaying current status of tasks |
US20020078204A1 (en) * | 1998-12-18 | 2002-06-20 | Dan Newell | Method and system for controlling presentation of information to a user based on the user's condition |
US20020083025A1 (en) * | 1998-12-18 | 2002-06-27 | Robarts James O. | Contextual responses based on automated learning techniques |
US20020080155A1 (en) * | 1998-12-18 | 2002-06-27 | Abbott Kenneth H. | Supplying notifications related to supply and consumption of user context data |
US20020087525A1 (en) * | 2000-04-02 | 2002-07-04 | Abbott Kenneth H. | Soliciting information based on a computer user's context |
US6480598B1 (en) * | 1997-09-16 | 2002-11-12 | Verizon Services Corp. | Methods and apparatus for automating the telephone operator services |
US6490698B1 (en) * | 1999-06-04 | 2002-12-03 | Microsoft Corporation | Multi-level decision-analytic approach to failure and repair in human-computer interactions |
US20030035532A1 (en) * | 2001-08-17 | 2003-02-20 | International Business Machines Corporation | Web-based distributed call center architecture |
US20030046401A1 (en) * | 2000-10-16 | 2003-03-06 | Abbott Kenneth H. | Dynamically determing appropriate computer user interfaces |
US20040005047A1 (en) * | 2002-07-05 | 2004-01-08 | Sbc Technology Resources, Inc. | Call routing from manual to automated dialog of interactive voice response system |
US20040098274A1 (en) * | 2002-11-15 | 2004-05-20 | Dezonno Anthony J. | System and method for predicting customer contact outcomes |
US6747675B1 (en) * | 1998-12-18 | 2004-06-08 | Tangis Corporation | Mediating conflicts in computer user's context data |
US6798876B1 (en) * | 1998-12-29 | 2004-09-28 | At&T Corp. | Method and apparatus for intelligent routing of incoming calls to representatives in a call center |
US6812937B1 (en) * | 1998-12-18 | 2004-11-02 | Tangis Corporation | Supplying enhanced computer user's context data |
US20050192936A1 (en) * | 2004-02-12 | 2005-09-01 | Meek Christopher A. | Decision-theoretic web-crawling and predicting web-page change |
Family Cites Families (17)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US6092725A (en) | 1997-01-24 | 2000-07-25 | Symbol Technologies, Inc. | Statistical sampling security methodology for self-scanning checkout system |
US6035104A (en) | 1996-06-28 | 2000-03-07 | Data Link Systems Corp. | Method and apparatus for managing electronic documents by alerting a subscriber at a destination other than the primary destination |
US7040541B2 (en) | 1996-09-05 | 2006-05-09 | Symbol Technologies, Inc. | Portable shopping and order fulfillment system |
US6837436B2 (en) | 1996-09-05 | 2005-01-04 | Symbol Technologies, Inc. | Consumer interactive shopping system |
US6409086B1 (en) | 1997-08-08 | 2002-06-25 | Symbol Technolgies, Inc. | Terminal locking system |
JPH11143488A (en) * | 1997-11-10 | 1999-05-28 | Hitachi Ltd | Speech recognizing device |
US6640214B1 (en) | 1999-01-16 | 2003-10-28 | Symbol Technologies, Inc. | Portable electronic terminal and data processing system |
US7010501B1 (en) | 1998-05-29 | 2006-03-07 | Symbol Technologies, Inc. | Personal shopping system |
SG79995A1 (en) * | 1998-11-02 | 2001-04-17 | Singtel Yellow Pages Pte Ltd | An automatic communications routing system and method |
US7133520B1 (en) * | 1999-04-27 | 2006-11-07 | Rockwell Electronic Commerce Technologies, Llc | Dynamic skill-based call routing |
US20010030664A1 (en) | 1999-08-16 | 2001-10-18 | Shulman Leo A. | Method and apparatus for configuring icon interactivity |
JP3608449B2 (en) * | 1999-09-09 | 2005-01-12 | 日本電信電話株式会社 | Voice response method and apparatus, and storage medium storing voice response program |
US6353398B1 (en) | 1999-10-22 | 2002-03-05 | Himanshu S. Amin | System for dynamically pushing information to a user utilizing global positioning system |
JP3444486B2 (en) * | 2000-01-26 | 2003-09-08 | インターナショナル・ビジネス・マシーンズ・コーポレーション | Automatic voice response system and method using voice recognition means |
JP3319457B2 (en) * | 2000-01-28 | 2002-09-03 | 三菱電機株式会社 | Telephone voice response system |
EP1176838A1 (en) | 2000-07-24 | 2002-01-30 | Hewlett-Packard Company, A Delaware Corporation | A communication platform for providing computer telephony integration services to remote subscribers, and associated method |
USD494584S1 (en) | 2002-12-05 | 2004-08-17 | Symbol Technologies, Inc. | Mobile companion |
-
2003
- 2003-06-30 US US10/610,486 patent/US20040264677A1/en not_active Abandoned
-
2004
- 2004-04-20 US US10/827,873 patent/US7742591B2/en not_active Expired - Fee Related
- 2004-05-21 JP JP2004152123A patent/JP2005027283A/en active Pending
- 2004-05-26 DE DE602004018875T patent/DE602004018875D1/en active Active
- 2004-05-26 AT AT04102315T patent/ATE420537T1/en not_active IP Right Cessation
- 2004-05-26 EP EP04102315A patent/EP1494499B1/en not_active Not-in-force
- 2004-05-27 BR BR0403831-2A patent/BRPI0403831A/en not_active Application Discontinuation
- 2004-05-28 CN CN2004100631081A patent/CN1592427B/en not_active Expired - Fee Related
- 2004-05-31 KR KR1020040039095A patent/KR101099231B1/en not_active IP Right Cessation
Patent Citations (51)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US4747054A (en) * | 1984-11-28 | 1988-05-24 | Conoco Inc. | Method for non-linear signal matching |
US5291550A (en) * | 1990-12-26 | 1994-03-01 | At&T Bell Laboratories | Dynamic network call distributor |
US5848131A (en) * | 1993-02-22 | 1998-12-08 | Murex Securities, Ltd. | Automatic information and routing system for telephonic services |
US5493692A (en) * | 1993-12-03 | 1996-02-20 | Xerox Corporation | Selective delivery of electronic messages in a multiple computer system based on context and environment of a user |
US5544321A (en) * | 1993-12-03 | 1996-08-06 | Xerox Corporation | System for granting ownership of device by user based on requested level of ownership, present state of the device, and the context of the device |
US5555376A (en) * | 1993-12-03 | 1996-09-10 | Xerox Corporation | Method for granting a user request having locational and contextual attributes consistent with user policies for devices having locational attributes consistent with the user request |
US5603054A (en) * | 1993-12-03 | 1997-02-11 | Xerox Corporation | Method for triggering selected machine event when the triggering properties of the system are met and the triggering conditions of an identified user are perceived |
US5611050A (en) * | 1993-12-03 | 1997-03-11 | Xerox Corporation | Method for selectively performing event on computer controlled device whose location and allowable operation is consistent with the contextual and locational attributes of the event |
US5812865A (en) * | 1993-12-03 | 1998-09-22 | Xerox Corporation | Specifying and establishing communication data paths between particular media devices in multiple media device computing systems based on context of a user or users |
US5561711A (en) * | 1994-03-09 | 1996-10-01 | Us West Technologies, Inc. | Predictive calling scheduling system and method |
US5519773A (en) * | 1994-06-07 | 1996-05-21 | Siemens Colm Communications Inc. | Call sharing for inbound and outbound call center agents |
US5530744A (en) * | 1994-09-20 | 1996-06-25 | At&T Corp. | Method and system for dynamic customized call routing |
US5896448A (en) * | 1996-12-17 | 1999-04-20 | Bellsouth Corporation | Method and apparatus for routing calls based on call success history |
US6480598B1 (en) * | 1997-09-16 | 2002-11-12 | Verizon Services Corp. | Methods and apparatus for automating the telephone operator services |
US5995805A (en) * | 1997-10-17 | 1999-11-30 | Lockheed Martin Missiles & Space | Decision-theoretic satellite communications system |
US6115462A (en) * | 1998-01-09 | 2000-09-05 | Gte Laboratories Incorporated | Method and apparatus for efficient call routing |
US20020083158A1 (en) * | 1998-12-18 | 2002-06-27 | Abbott Kenneth H. | Managing interactions between computer users' context models |
US6747675B1 (en) * | 1998-12-18 | 2004-06-08 | Tangis Corporation | Mediating conflicts in computer user's context data |
US20010043231A1 (en) * | 1998-12-18 | 2001-11-22 | Abbott Kenneth H. | Thematic response to a computer user's context, such as by a wearable personal computer |
US20050034078A1 (en) * | 1998-12-18 | 2005-02-10 | Abbott Kenneth H. | Mediating conflicts in computer user's context data |
US6842877B2 (en) * | 1998-12-18 | 2005-01-11 | Tangis Corporation | Contextual responses based on automated learning techniques |
US20020052930A1 (en) * | 1998-12-18 | 2002-05-02 | Abbott Kenneth H. | Managing interactions between computer users' context models |
US20020054174A1 (en) * | 1998-12-18 | 2002-05-09 | Abbott Kenneth H. | Thematic response to a computer user's context, such as by a wearable personal computer |
US6812937B1 (en) * | 1998-12-18 | 2004-11-02 | Tangis Corporation | Supplying enhanced computer user's context data |
US20020078204A1 (en) * | 1998-12-18 | 2002-06-20 | Dan Newell | Method and system for controlling presentation of information to a user based on the user's condition |
US20010040590A1 (en) * | 1998-12-18 | 2001-11-15 | Abbott Kenneth H. | Thematic response to a computer user's context, such as by a wearable personal computer |
US20020083025A1 (en) * | 1998-12-18 | 2002-06-27 | Robarts James O. | Contextual responses based on automated learning techniques |
US20020080155A1 (en) * | 1998-12-18 | 2002-06-27 | Abbott Kenneth H. | Supplying notifications related to supply and consumption of user context data |
US20020080156A1 (en) * | 1998-12-18 | 2002-06-27 | Abbott Kenneth H. | Supplying notifications related to supply and consumption of user context data |
US6801223B1 (en) * | 1998-12-18 | 2004-10-05 | Tangis Corporation | Managing interactions between computer users' context models |
US20020099817A1 (en) * | 1998-12-18 | 2002-07-25 | Abbott Kenneth H. | Managing interactions between computer users' context models |
US6466232B1 (en) * | 1998-12-18 | 2002-10-15 | Tangis Corporation | Method and system for controlling presentation of information to a user based on the user's condition |
US20010040591A1 (en) * | 1998-12-18 | 2001-11-15 | Abbott Kenneth H. | Thematic response to a computer user's context, such as by a wearable personal computer |
US6791580B1 (en) * | 1998-12-18 | 2004-09-14 | Tangis Corporation | Supplying notifications related to supply and consumption of user context data |
US20010043232A1 (en) * | 1998-12-18 | 2001-11-22 | Abbott Kenneth H. | Thematic response to a computer user's context, such as by a wearable personal computer |
US6798876B1 (en) * | 1998-12-29 | 2004-09-28 | At&T Corp. | Method and apparatus for intelligent routing of incoming calls to representatives in a call center |
US6490698B1 (en) * | 1999-06-04 | 2002-12-03 | Microsoft Corporation | Multi-level decision-analytic approach to failure and repair in human-computer interactions |
US20020032689A1 (en) * | 1999-12-15 | 2002-03-14 | Abbott Kenneth H. | Storing and recalling information to augment human memories |
US6549915B2 (en) * | 1999-12-15 | 2003-04-15 | Tangis Corporation | Storing and recalling information to augment human memories |
US20030154476A1 (en) * | 1999-12-15 | 2003-08-14 | Abbott Kenneth H. | Storing and recalling information to augment human memories |
US6513046B1 (en) * | 1999-12-15 | 2003-01-28 | Tangis Corporation | Storing and recalling information to augment human memories |
US20020087525A1 (en) * | 2000-04-02 | 2002-07-04 | Abbott Kenneth H. | Soliciting information based on a computer user's context |
US20020044152A1 (en) * | 2000-10-16 | 2002-04-18 | Abbott Kenneth H. | Dynamic integration of computer generated and real world images |
US20020054130A1 (en) * | 2000-10-16 | 2002-05-09 | Abbott Kenneth H. | Dynamically displaying current status of tasks |
US20030046401A1 (en) * | 2000-10-16 | 2003-03-06 | Abbott Kenneth H. | Dynamically determing appropriate computer user interfaces |
US20030035532A1 (en) * | 2001-08-17 | 2003-02-20 | International Business Machines Corporation | Web-based distributed call center architecture |
US20040005047A1 (en) * | 2002-07-05 | 2004-01-08 | Sbc Technology Resources, Inc. | Call routing from manual to automated dialog of interactive voice response system |
US6807274B2 (en) * | 2002-07-05 | 2004-10-19 | Sbc Technology Resources, Inc. | Call routing from manual to automated dialog of interactive voice response system |
US20050041796A1 (en) * | 2002-07-05 | 2005-02-24 | Sbc Technology Resources, Inc. | Call routing from manual to automated dialog of interactive voice response system |
US20040098274A1 (en) * | 2002-11-15 | 2004-05-20 | Dezonno Anthony J. | System and method for predicting customer contact outcomes |
US20050192936A1 (en) * | 2004-02-12 | 2005-09-01 | Meek Christopher A. | Decision-theoretic web-crawling and predicting web-page change |
Cited By (71)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US7624007B2 (en) * | 1999-11-12 | 2009-11-24 | Phoenix Solutions, Inc. | System and method for natural language processing of sentence based queries |
US10560579B1 (en) | 2002-03-07 | 2020-02-11 | Wai Wu | Intelligent communication routing |
US9736308B1 (en) | 2002-03-07 | 2017-08-15 | Wai Wu | Intelligent communication routing |
US11790413B2 (en) | 2003-02-05 | 2023-10-17 | Hoffberg Family Trust 2 | System and method for communication |
US10943273B2 (en) | 2003-02-05 | 2021-03-09 | The Hoffberg Family Trust 2004-1 | System and method for determining contingent relevance |
US9860391B1 (en) | 2003-03-07 | 2018-01-02 | Wai Wu | Method and system for matching entities in an auction |
US10237420B1 (en) | 2003-03-07 | 2019-03-19 | Wai Wu | Method and system for matching entities in an auction |
US20050055216A1 (en) * | 2003-09-04 | 2005-03-10 | Sbc Knowledge Ventures, L.P. | System and method for the automated collection of data for grammar creation |
US20050069122A1 (en) * | 2003-09-30 | 2005-03-31 | Xiaofan Lin | System and method for operator assisted automated call handling |
US7844045B2 (en) * | 2004-06-16 | 2010-11-30 | Panasonic Corporation | Intelligent call routing and call supervision method for call centers |
US20050286705A1 (en) * | 2004-06-16 | 2005-12-29 | Matsushita Electric Industrial Co., Ltd. | Intelligent call routing and call supervision method for call centers |
US9368111B2 (en) | 2004-08-12 | 2016-06-14 | Interactions Llc | System and method for targeted tuning of a speech recognition system |
US8751232B2 (en) | 2004-08-12 | 2014-06-10 | At&T Intellectual Property I, L.P. | System and method for targeted tuning of a speech recognition system |
US9112972B2 (en) | 2004-12-06 | 2015-08-18 | Interactions Llc | System and method for processing speech |
US9350862B2 (en) | 2004-12-06 | 2016-05-24 | Interactions Llc | System and method for processing speech |
US7756263B1 (en) * | 2004-12-23 | 2010-07-13 | Sprint Spectrum L.P. | Method and system for timed interaction with an interactive voice response system |
US7933400B2 (en) | 2004-12-23 | 2011-04-26 | Sprint Spectrum L.P. | Method and system for timed interaction with an interactive voice response system |
US7436948B1 (en) * | 2004-12-23 | 2008-10-14 | Sprint Spectrum L.P. | Method and system for timed interaction with an interactive voice response |
US20100220842A1 (en) * | 2004-12-23 | 2010-09-02 | Sprint Spectrum L.P. | Method and System for Timed Interaction with an Interactive Voice Response System |
US8600757B2 (en) * | 2005-01-07 | 2013-12-03 | At&T Intellectual Property Ii, L.P. | System and method of dynamically modifying a spoken dialog system to reduce hardware requirements |
US9088652B2 (en) | 2005-01-10 | 2015-07-21 | At&T Intellectual Property I, L.P. | System and method for speech-enabled call routing |
US8824659B2 (en) | 2005-01-10 | 2014-09-02 | At&T Intellectual Property I, L.P. | System and method for speech-enabled call routing |
US8280030B2 (en) | 2005-06-03 | 2012-10-02 | At&T Intellectual Property I, Lp | Call routing system and method of using the same |
US8619966B2 (en) | 2005-06-03 | 2013-12-31 | At&T Intellectual Property I, L.P. | Call routing system and method of using the same |
US8175253B2 (en) * | 2005-07-07 | 2012-05-08 | At&T Intellectual Property I, L.P. | System and method for automated performance monitoring for a call servicing system |
US20070025528A1 (en) * | 2005-07-07 | 2007-02-01 | Sbc Knowledge Ventures, L.P. | System and method for automated performance monitoring for a call servicing system |
USRE49334E1 (en) | 2005-10-04 | 2022-12-13 | Hoffberg Family Trust 2 | Multifactorial optimization system and method |
US10567975B2 (en) | 2005-10-04 | 2020-02-18 | Hoffberg Family Trust 2 | Multifactorial optimization system and method |
US8095371B2 (en) * | 2006-02-20 | 2012-01-10 | Nuance Communications, Inc. | Computer-implemented voice response method using a dialog state diagram to facilitate operator intervention |
US20070198272A1 (en) * | 2006-02-20 | 2007-08-23 | Masaru Horioka | Voice response system |
US8923506B1 (en) | 2006-02-28 | 2014-12-30 | Allstate Insurance Company | Systems and methods for automated call-handling and processing |
US8494152B1 (en) * | 2006-02-28 | 2013-07-23 | Allstate Insurance Company | Systems and methods for automated call-handling and processing |
US11792318B2 (en) | 2006-02-28 | 2023-10-17 | Allstate Insurance Company | Systems and methods for automated call-handling and processing |
US10129399B1 (en) | 2006-02-28 | 2018-11-13 | Allstate Insurance Company | Systems and methods for automated call-handling and processing |
US11431845B1 (en) | 2006-02-28 | 2022-08-30 | Allstate Insurance Company | Systems and methods for automated call-handling and processing |
US9674352B1 (en) | 2006-02-28 | 2017-06-06 | Allstate Insurance Company | Systems and methods for automated call-handling and processing |
US10778844B1 (en) | 2006-02-28 | 2020-09-15 | Allstate Insurance Company | Systems and methods for automated call-handling and processing |
US10491748B1 (en) | 2006-04-03 | 2019-11-26 | Wai Wu | Intelligent communication routing system and method |
US8300798B1 (en) * | 2006-04-03 | 2012-10-30 | Wai Wu | Intelligent communication routing system and method |
US9807239B1 (en) | 2006-04-03 | 2017-10-31 | Wai Wu | Intelligent communication routing system and method |
US9549065B1 (en) * | 2006-05-22 | 2017-01-17 | Convergys Customer Management Delaware Llc | System and method for automated customer service with contingent live interaction |
US8379830B1 (en) * | 2006-05-22 | 2013-02-19 | Convergys Customer Management Delaware Llc | System and method for automated customer service with contingent live interaction |
US20080077470A1 (en) * | 2006-09-26 | 2008-03-27 | Fujitsu Limited | Respondent-information output apparatus, respondent-information output method, and computer product |
US8204205B1 (en) * | 2007-02-08 | 2012-06-19 | At&T Intellectual Property Ii, L.P. | Method for increasing the efficiency of automated telephony systems |
US8532279B2 (en) | 2007-02-08 | 2013-09-10 | At&T Intellectual Property Ii, L.P. | Method for increasing the efficiency of automated telephony systems |
US8886645B2 (en) * | 2007-10-15 | 2014-11-11 | Chacha Search, Inc. | Method and system of managing and using profile information |
US20090100047A1 (en) * | 2007-10-15 | 2009-04-16 | Chacha Search, Inc. | Method and system of managing and using profile information |
US8601030B2 (en) * | 2011-09-09 | 2013-12-03 | International Business Machines Corporation | Method for a natural language question-answering system to complement decision-support in a real-time command center |
US9213695B2 (en) | 2012-02-06 | 2015-12-15 | Language Line Services, Inc. | Bridge from machine language interpretation to human language interpretation |
US9213693B2 (en) | 2012-04-03 | 2015-12-15 | Language Line Services, Inc. | Machine language interpretation assistance for human language interpretation |
US20130329880A1 (en) * | 2012-06-11 | 2013-12-12 | Interactive Intelligence, Inc. | Method and system for improving the productivity of calling agents and call yield |
WO2013188185A1 (en) * | 2012-06-11 | 2013-12-19 | Interactive Intelligence, Inc. | Method and system for improving the productivity of calling agents and call yield |
US10515156B2 (en) * | 2012-08-31 | 2019-12-24 | Verint Americas Inc | Human-to-human conversation analysis |
US11455475B2 (en) * | 2012-08-31 | 2022-09-27 | Verint Americas Inc. | Human-to-human conversation analysis |
US9781270B2 (en) | 2014-08-01 | 2017-10-03 | Genesys Telecommunications Laboratories, Inc. | System and method for case-based routing for a contact |
US10171669B2 (en) | 2014-08-01 | 2019-01-01 | Genesys Telecommunications Laboratories, Inc. | System and method for routing interactions for a contact center based on intelligent and dynamic routing considerations |
US9848084B2 (en) | 2014-08-01 | 2017-12-19 | Genesys Telecommunications Laboratories, Inc. | Adaptable business objective routing for a contact center |
WO2016019194A1 (en) * | 2014-08-01 | 2016-02-04 | Genesys Telecommunications Laboratories, Inc. | System and method for case-based routing for a contact center |
US9473632B2 (en) | 2014-08-27 | 2016-10-18 | Avaya Inc. | System and method to improve self-service experience and optimize contact center resources |
US20160127553A1 (en) * | 2014-10-31 | 2016-05-05 | Avaya Inc. | System and method for managing resources of an enterprise |
US11621932B2 (en) * | 2014-10-31 | 2023-04-04 | Avaya Inc. | System and method for managing resources of an enterprise |
US10718031B1 (en) | 2014-11-03 | 2020-07-21 | Wai Wu | Method and system for matching entities in an auction |
WO2016167424A1 (en) * | 2015-04-16 | 2016-10-20 | 주식회사 플런티코리아 | Answer recommendation device, and automatic sentence completion system and method |
US10230846B2 (en) * | 2016-05-17 | 2019-03-12 | Facebook, Inc. | Systems and methods for interacting with answering systems |
US11270686B2 (en) * | 2017-03-28 | 2022-03-08 | International Business Machines Corporation | Deep language and acoustic modeling convergence and cross training |
US20180286386A1 (en) * | 2017-03-28 | 2018-10-04 | International Business Machines Corporation | Deep language and acoustic modeling convergence and cross training |
US11861316B2 (en) | 2018-05-02 | 2024-01-02 | Verint Americas Inc. | Detection of relational language in human-computer conversation |
US11822888B2 (en) | 2018-10-05 | 2023-11-21 | Verint Americas Inc. | Identifying relational segments |
US11250213B2 (en) * | 2019-04-16 | 2022-02-15 | International Business Machines Corporation | Form-based transactional conversation system design |
US11250215B2 (en) * | 2019-04-16 | 2022-02-15 | International Business Machines Corporation | Form-based transactional conversation system design |
US11316980B2 (en) | 2019-11-26 | 2022-04-26 | International Business Machines Corporation | Agent to bot transfer |
Also Published As
Publication number | Publication date |
---|---|
KR101099231B1 (en) | 2011-12-27 |
CN1592427B (en) | 2011-01-26 |
DE602004018875D1 (en) | 2009-02-26 |
EP1494499B1 (en) | 2009-01-07 |
ATE420537T1 (en) | 2009-01-15 |
US7742591B2 (en) | 2010-06-22 |
EP1494499A3 (en) | 2005-06-15 |
BRPI0403831A (en) | 2005-06-07 |
CN1592427A (en) | 2005-03-09 |
KR20050004702A (en) | 2005-01-12 |
EP1494499A2 (en) | 2005-01-05 |
JP2005027283A (en) | 2005-01-27 |
US20040264672A1 (en) | 2004-12-30 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US20040264677A1 (en) | Ideal transfer of call handling from automated systems to human operators based on forecasts of automation efficacy and operator load | |
US9871927B2 (en) | Complexity aware call-steering strategy in heterogeneous human/machine call-center environments | |
US10171659B2 (en) | Customer portal of an intelligent automated agent for a contact center | |
CA2917294C (en) | Intelligent automated agent for a contact center | |
US11159679B2 (en) | Automated systems and methods for natural language processing with speaker intention inference | |
US20230289702A1 (en) | System and Method of Assigning Customer Service Tickets | |
JP2023530549A (en) | Systems and methods for conducting automated interview sessions | |
KR20180071312A (en) | Routing interactions with optimized contact center agents based on machine learning | |
US20190354874A1 (en) | Platform for autonomous and goal-driven digital assistants | |
US8831208B2 (en) | System and method for optimizing call flows of a spoken dialog system | |
Horvitz et al. | Complementary computing: policies for transferring callers from dialog systems to human receptionists | |
US20210390491A1 (en) | Coach-Assist Controller for Customer Service Representative (CSR) Interactions | |
Natarajan et al. | Speech-enabled natural language call routing: BBN Call Director | |
US7908143B2 (en) | Dialog call-flow optimization | |
KR102306953B1 (en) | Method of medical support based on voice recognition | |
Avdagić-Golub et al. | Optimization of agent-user matching process using a machine learning algorithms | |
Hunt et al. | Artificial Intelligence for Customer Experience and Care | |
WO2019187019A1 (en) | Customer treatment assistance system and customer treatment assistance method | |
CN116580529A (en) | State reminding method and device | |
Natarajan et al. | Natural Language Call Routing with BBN Call Director |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
AS | Assignment |
Owner name: MICROSOFT CORPORATION, WASHINGTON Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:HORVITZ, ERIC J.;PAEK, TIMOTHY S.;REEL/FRAME:014284/0136 Effective date: 20030630 |
|
STCB | Information on status: application discontinuation |
Free format text: ABANDONED -- FAILURE TO RESPOND TO AN OFFICE ACTION |
|
AS | Assignment |
Owner name: MICROSOFT TECHNOLOGY LICENSING, LLC, WASHINGTON Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:MICROSOFT CORPORATION;REEL/FRAME:034766/0001 Effective date: 20141014 |