In Japanese society, customers are king. However, the virulence of some of these customers is a real plague. To protect employees who handle these calls, an increasing number of companies are turning to generative artificial intelligence to soften the voices of aggressive customers, reports the country’s economic daily, “Nihon Keizai Shimbun.”
“Be more accommodating, or I’ll report you on social media! I’ll say you’re a terrible company that doesn’t care about its customers!”
The voice, so aggressive that it forces me to grimace, echoes in one of Softbank’s meeting rooms. It is a demonstration recording made by an actress playing an angry customer.
By activating the “anger suppression” feature, the same shrill voice is instantly replaced by a softer, almost childlike tone. With the discomfort thus reduced, the subject of the complaint becomes easier to grasp.
This technology, developed by Softbank, uses AI to synthesize a voice and remove its threatening tone. It is set to be commercialized for call centers by the end of 2025.
A leading figure in voice synthesis, Shinnosuke Takamichi, a researcher at the University of Tokyo, contributed to its development. Trained with over 10,000 voice samples, recordings of male and female voices of various ages in different environments, the AI has learned to distinguish the characteristics of emotionally charged speech from neutral speech. It is capable of eliminating the oppressive tone of a voice to make it calmer and less aggressive. Anticipating its use in call centers, efforts have been made to reduce the gap between the real voice and the voice processed by AI.
“Protecting Employees’ Emotional Well-Being”
“Listening to customer complaints all day represents a huge mental burden. This measure aims to protect the emotional health of employees,” explains Toshiyuki Nakatani, head of the IT and design division.
By comparing the modified voice with the original voice, the company’s operators observed a decrease (up to 37%) in perceived aggression. “We will improve the system so that it can function in noisy environments and with multiple interlocutors,” adds Nakatani.
The issue of customer harassment, known as kasuhara in Japanese—a term derived from the English expression “customer harassment”—has worsened in recent years, especially in the service sector. According to a survey by the Ministry of Labor, 28% of companies reported receiving reports of customer harassment in the past three years, the highest percentage after power abuse (64%) and sexual harassment (40%).
With the rise of e-commerce, the use of call centers is increasing, as is the number of harassment cases involving operators. This has prompted companies in the sector to take measures to protect their staff. Last March, major call centers added a provision regarding kasuhara to their workplace anti-harassment rules, as it has become a matter of public interest.
Feedback and Recommendations
Among them, some are also using real-time transcription of calls through generative AI to prevent issues. The exchanges with customers, thus transcribed, are continuously monitored by a center manager, who immediately alerts their superiors when unacceptable behavior, such as insults, is identified.
Some companies are also planning to use AI for employee training. Fujitsu, in collaboration with Professor Masayuki Kiryu, a criminal psychology researcher at Toyo University, has created a tool that simulates kasuhara. By conversing with an AI that incorporates conversation models from difficult customers, employees learn how to manage crises.
AI is also used to assess the experience. Based on a pre-questionnaire and vital data such as heart rate and respiratory frequency measured during the simulation, it evaluates the operator. The AI avatar then provides practical advice depending on the psychological characteristics of each employee, such as: “Clearly indicate to your interlocutor that you heard what they said.” Fujitsu and Toyo University plan to test the effectiveness of this tool with employees responsible for call center operations.
Preventing Employee Turnover
On its part, the conversational AI developer IVRy offers small and medium-sized enterprises an automatic phone answering service. The AI instantly summarizes the customer’s statements and relays them to the company. Businesses can easily design their own automatic response system from a computer or smartphone. An increasing number of SMEs, especially restaurants, are adopting this service to streamline call handling. “It prevents employees from quitting due to absurd requests during peak times,” states a company representative.
Public authorities are also taking action against customer harassment. In 2022, the Ministry of Labor published a manual for companies on how to address it. It includes examples of responses to verbal aggression, as well as guidelines, particularly for establishing a consultation system in case of harm. The Tokyo Metropolitan Government is also preparing to implement its first municipal ordinance on customer harassment.
If no action is taken, customer harassment can lead to employees quitting or being unable to continue working. For the call center sector to develop sustainably, it is urgent to create an environment where operators can engage with customers as equals.
