Fujitsu Laboratories is putting artificial intelligence (AI) to work, identifying voice patterns tied to customers' satisfaction or dissatisfaction during their calls.

Rather than formally gather information about customers' level of satisfaction through user surveys or through speech recognition, converting speech to text and then analysing what's being said, the technology takes into account the average pitch of the voice, its degree of variation and also characteristic changes at relative points within voice data that covers multiple words, such as the start or end of speech.

Fujitsu voice AI 01 Figure 1: Quantification of voice cheerfulness through pitch analysis. (Source: Fujitsu)

A customer's "voice cheerfulness" emerges from patterns of changes in voice pitch, usually at a high tone or when the voice's tone and volume change a great deal, explained the researchers. From these changes of pitch, proprietary conversion algorithms are able to identify the unique characteristics of a cheerful voice, especially at the beginnings and endings of conversational statements. The perceived voice cheerfulness correlates well to the degree of customer satisfaction, Fujitsu Laboratories claimed. Combining this with customer-service evaluations, and using machine learning, a threshold point between satisfaction and dissatisfaction can be set, meaning that just by listening in, AI software could identify on-the-fly when a customer is satisfied or dissatisfied.

Fujitsu voice AI 02 Figure 2: AI listens for "voice cheerfulness."

Users of such technology could adopt their marketing or conversational tactics on the fly, too.

In a field trial using this technology in Fujitsu Limited and Fujitsu FSAS Inc. call centres, Fujitsu reported increases in the efficiency of training, such as monitoring and evaluation of support personnel and feedback on the results, reducing the required training time by about 30%. Being AI-based and more objective, the evaluation methodology received a greater degree of acceptance by both evaluators and trainees.

Fujitsu voice AI 03 Figure 3: A more objective and efficient training for service personnel.

In the future, this technology could be incorporated into Fujitsu's Human Centric AI Zinrai, to be offered as a product for use in its digital solutions and services, such as automatic interactive answering services.