Ivars Bušmanis, Latvijas Avīze
Artificial intelligence cloned voice of a friend – the latest scam.
“Today I received a call from an unknown number, I picked up the phone and a close friend of mine was crying and said he was in a lot of trouble – he had been in an accident. It was his fault, he hit a girl – now he is in hospital with broken ribs… Of course, I am upset and say that it is terrible – ask how I can help, because today I am outside Riga – in Koknese. As I was upset, I forgot that the call was from an unknown number. The friend says he urgently needs EUR 25 000 to avoid a criminal case. I see right through it, I have heard and read this hundreds of times. I say: “I will call you right back!” I immediately call my friend – turns out that he is all right. I have not yet finished talking to my friend when I receive an incoming call from another unknown number. I pick up. Again the “friend” is desperate and reproachful – “why did you hang up, I am in so much trouble!” I reply – both of these numbers will be handed over to the police immediately. The call drops.
The worst thing – a “friend” calling from both numbers had the voice of my real friend!”
Jānis Karalis, a Board Member of Kokneses fonds, i.e., the Head of the well-known Likteņdārzs, shares this on Facebook about the call he received on Saturday.
Although he only reached his mobile operator Bite in the evening (after listening to music for 15 minutes on the answering machine), Jānis Karalis was “both thanked and asked to name the two numbers, when he received the calls, and was told that such a scheme is becoming common in the USA”.
Alarms going off before a fire
On Monday, Bite Latvija issued a press release saying that phone scammers have started using artificial intelligence (AI) to call people and use the voices of people they know, citing the above case. “Yes, AI tools are very successful in imitating the voice of a specific person in English, but in Latvian not so good,” Reinis Pudāns, Head of Internal Security Processes at Bite Latvija, told Latvijas Avīze.
Bite Latvija already informed people in January to be careful because there is a new type of phone scam – robot calls in Latvian, which are made using AI voice cloning technology.
At that time, the State Police failed to collect evidence of AI being used in the scam. This time too, the police and Bite security expert doubt whether AI was used.
So far, neither LMT nor Tele2 have received any complaints from customers about imitated voices of relatives. Is the alarm for nothing? Definitely not. Such scams are very common in major foreign languages. The fact that Bite’s new anti-scam solution blocked 10 000 suspicious calls in June and 20 000 in July is already evidence of the activity of scammers.
Where have these calls come from? Reinis Pudāns tells Latvijas Avīze that Bite considers calls disguised as Latvian numbers to be potentially suspicious. First of all, calls from Latvian mobile and fixed network numbers that are not assigned for use by any operator, such as 11111 or that do not begin with 2 or 6. Second, calls from Bite Latvija mobile network numbers that are made from abroad but which, according to the system, are not located abroad. Third, calls from Latvian fixed network numbers that are made from abroad, e.g. the scammers allegedly made such calls from abroad from Swedbank fixed number. Initially, 112 and police numbers were also used, which was quickly prevented.
When a call comes in from abroad, each operator in Latvia checks its own numbers to see if they are really abroad, but… does not and cannot check calls from customers of other operators.
Most scam calls come through Tet and Tele2.
LMT has also developed a system to block scam calls from foreign numbers and Latvian numbers that have not been assigned to any user and has introduced protection against attempts to use official authorities’ and banks’ numbers for scam calls. “We are setting up an initiative for other operators to join our idea of a single database (similar to numuri.lv) where all active numbers and those in roaming (scam calls initially come from foreign channels) can be found online,” says Valdis Jalinskis, Head of Public Relations at LMT.
In Estonia, the law already requires this. Lithuania will follow early next year. But the Latvian regulator, the Public Utilities Commission, has not gone further than consultations.
The only thing the regulator and mobile operators will not be able to prevent is the incoming internet calls: Whatsapp, Telegram and other apps have encrypted calls and it is not in the power of operators to prevent scam,
mentions Valdis Jalinskis. “It is important to keep a critical mindset.”
Where to get a friend’s voice?
For several years now, we have been hearing various stories abroad about successful voice imitations. So-called deep-fakes are now being used in political battles, for example now, before the US election, US President Joe Biden has had words put in his mouth that he did not say, or support has been announced for Donald Trump from people who have never expressed it. These are big political stakes, but on a daily basis, voice imitation is mainly used for money scams.
Last year, Gary Schildhorn, an attorney experienced in scams, almost became a victim himself. He confided his story to CBS News. His son Brett called: “Dad, I am in trouble, I got in an accident. I think I broke my nose. I hit a pregnant woman. They arrested me, I am in jail. You need to help me.” Gary never doubted for a moment that it was his son. Within two minutes, he received a call from his lawyer and a court clerk confirming what had happened.
On his way to the bank to get the 9000 dollars in cash, he called his son again.
He then realised that his son was fine and that he had been scammed. As Gary Schildhorn had not lost any money, no criminal proceedings were initiated, and the scammers could not be caught because they were using phones with prepaid cards.
Any internet search engine can find various offers for voice imitation in a few seconds, such as ElevenLabs, Murf AI or LOVO. All it takes is a 30-second audio file with a voice recording, and a short while later the AI can make that voice read out any text. For a monthly fee of five US dollars.
The voice recording can be taken from a social media video or by recording a phone call. However, these sites require proof that the consent of the owners of these voices has been obtained. However, this can easily be circumvented or posed as your own voice. Any of us can do it in a few minutes.
It is more difficult to have a conversation in real time, but this can be done in the paid version of ChatGPT, Reinis Pudāns reluctantly tells Latvijas Avīze. You can also enter the number you want. Thus, it is technically already possible to call in a friend’s voice from a friend’s phone number. “Technology makes it possible to put any number as a caller in a central station, but this requires a very large investment,” notes Valdis Jalinskis.
“In this particular case, it is a very complex and resource-intensive form of scam, as it requires both the relevant AI solution, as well as a voice recording to synthesise and, for example, the phone number of a friend or family member.
In theory, voice recordings can be obtained on social networks, as people share videos of themselves where their voice can be heard. However, this is probably the simplest element of this scam, as you still need to obtain the relevant phone numbers, i.e. the recipient of the calls must be a person who recognises a friend or relative in the voice synthesised by AI,” comments Oskars Fīrmanis, Head of Public Relations at Tele2 Latvia.
Is it that hard? I quickly found the phone number of Jānis Karalis on the website of Likteņdārzs and used it to call him. But how to get a friend’s voice? “My friend did call in a voice that was almost crying, and I had seen him like that once in my life. There was no doubt in my mind that it was him. The fact that it was from a strange number was not confusing. In accidents there are situations when you have to call from other numbers. It was only when they started asking for 25 000 that I got suspicious,” Jānis Karalis tells me.
Presumably, the scammers had already found out about Jānis’ friends from Facebook. Can the number of a friend also be found on the internet? Yes, he is the manager of a company whose number can be found on the website, confirms Jānis.
And one more detail, this time important, which casts doubt on whether AI was used. The friend in question, who was supposedly imitated, is Russian by nationality and the call was in Russian. In this language, the AI is sufficiently developed for the voice to be reliably imitated.
How did the scammers get the voice sample? A friend confirmed to Jānis that he had indeed received calls from unknown people that week, but that he had kept his speech short, in fact saying that he did not have time to talk. That seems to have been enough. So Jānis Karalis warns:
“Please take care yourself and warn your seniors: if someone calls from an unknown number in your voice, please end the call immediately and call the real number to clarify the situation!
Protect yourself and your loved ones!”
But in the near future, we may hear scammers calling our loved ones from their numbers, so we will need to be able to counteract AI with our natural intelligence in future scam attempts: not sharing sensitive data over the phone, using passwords when talking about money with people we know, asking follow-up questions about things we both know.
