Is your phone listening to you to sell personalized ads?

Anyone who’s ever had a conversation with a friend—and then been served with online ads for the topic they were discussing—may have found the coincidence a little too hard to overlook. 

Meta, which owns Facebook and Instagram, insists it doesn’t use the microphones on personal devices to pick up conversations and influence ads. 

“We understand that sometimes ads can be so specific, it seems like we must be listening to your conversations through your microphone, but we're not,” says the social media giant.  

Data experts have attempted to debunk the rumor by explaining personalized ads as behavioral advertising

The debate has been rumbling on for several years. Data experts have attempted to debunk the rumor by explaining personalization occurs thanks to the vast amounts of information that can be collected from browsing histories and geolocation tools—a practice known as behavioral advertising. 

However, new research from Esade’s David López-López and co-authors from two other top Spanish universities has revealed that the suspicions are well-founded. 

Mobile devices, the researchers conclude, listen to conversations through microphones and create personalized ads based on what the person wants or has done. 

The new age of digital marketing  

Online user-generated content has transformed the digital marketing landscape. The use of personal devices including mobile phones, tablets and wearable technology, combined with the data collected by search engines and social networks, leaves a detailed digital footprint.  

This vast amount of data allows marketing companies to classify behavior, detect trends and discover individual preferences. The information, collected and processed in conjunction with AI tools, is used to anticipate consumer behavior and has facilitated the use of new market formulas such as pricing algorithms and personalized processes.  

The tourism industry is one of the sectors most affected by this relatively new way of marketing to consumers. As digital citizens, consumers adapt their behavior and decisions in accordance with the marketing tools brands use. Travelers in particular are highly likely to decide which destinations and attractions to visit based on online content generated by other users. 

Bypassing privacy regulations 

As these practices have become increasingly widespread, the value to brands and companies of personalizing the online experience has grown exponentially. The collection of consumer data has become so integral to marketing that ethical lines are now blurred, with some companies prepared to cross privacy boundaries to collect it. 

Regulations that attempt to prevent this unethical behavior have little impact. Antivirus software company Avast says that 60 percent of Spaniards accept website cookie policies that record browsing activities and give consent to internet platforms to distribute the data. Only 13 percent reject the cookies, and 14 percent read the privacy policy. 

The collection of consumer data has become so integral to marketing that ethical lines are now blurred

Video conferencing applications require users to give permission for platforms to access their camera and microphone. In 2022, researchers from the University of Wisconsin-Madison and Loyola University Chicago discovered that the Webex platform was listening to users via their device microphones, even if they were muted. 

Parent company Cisco responded that it collected data to support the user experience, before announcing it would stop gathering information from muted microphones. 

Google’s information auctions 

Real-Time Bidding (RTB) is a Google initiative that allows the market-dominant search platform to collect information, via the use of cookies, including the tastes, geographic location and consumption habits of users. The information is sold to advertisers via a bid-based, real-time auction. 

The Irish Council of Civil Liberties says that European citizens receive an average of 376 RTB ‘broadcasts’ (or information collection points) every day. In the United States, users receive an average of 747 broadcasts per user, per day. In 2021, these auctions generated a revenue of US$117 billion. 

Against this backdrop of known data collection methods, the researchers conducted a scientific observation study of the advertisements received by tourists. Using a series of 10 tests, they analyzed whether 27 participants received ads related to a topic that had been discussed near a mobile phone for a certain amount of time. 

Collecting conversations 

During the supervised studies, a group of no more than three people maintained a fluent conversation of exactly 10 minutes using written prompts given to them by the researchers in sealed envelopes. Conversations took place within a one-meter distance of a mobile phone with the microphone activated.  

Participants discussed topics such as “journey to Rome,” “trip to Budapest” and “journey to New York”. Keywords including local landmarks (“Rome Coliseum”), available accommodation (“tourist apartments center of Rome”) or activities on offer (“eating pizza”) were expressly mentioned during each conversation. 

After each conversation, participants browsed social media and internet pages and were asked to screenshot any mentions of the topics or keywords discussed within the next five days. 

Yes, your phone is recording you 

In 100 percent of the observations, at least one ad related to the general subject discussed (the tourism sector) was received. In half of the tests, users received an ad specifically related to the specific destination being discussed. In three of the ten tests, ads were presented related to how to travel to the area or tourist attraction. In two cases, the same ad was repeated during the same test but for different participants and at different times. 

The results confirm the urgent need to legislate on the recording of private conversations and their commercial use

The channel with the most relevant ads was Instagram (probably due to its growth and high levels of advertising content, the researchers say), with 69 instances. Twitter showed eight ads, Facebook five, and RAC1 and Emisora.org websites two each. YouTube, Cadena SER, Marca, Sport, La Vanguardia, Spotify and Vandal each served one relevant ad. 

Tourist destinations made up 35.6 percent of the ads, transport 22 percent, and accommodation 18.6 percent. Ads corresponding to excursions and activities were 8.5 percent of the total, with tourist attractions at 7.6 percent, travel agencies at 5.9 percent and hotels at 1.7 per cent. 

The results, say the researchers, confirm the urgent need to legislate on the recording of private conversations and their commercial use. And, although they acknowledge the limited scale of the study, recently publicized cases of dubious or illegal recordings being made without owners’ consent should bring the issue to the forefront of the media.  

When combined with the phenomena of fake news, they conclude, there is a huge amount of potential for scientific research to address the misuse of information and protect consumers.  

And if that isn’t enough to dissuade businesses from breaching privacy regulations, perhaps the researchers’ findings that the ads cause such discomfort to users that they negatively impact the brand being advertised might persuade them to change.