Facebook removes inauthentic assets using AI-generated faces in Argentina

Network used fake profiles to create fictitious personas supporting Argentina’s Kirchnerism political movement

(Source: @estebanpdl/DFRLab)

On December 22, 2020, Facebook took down an Argentinian network of 663 Facebook accounts and 388 Instagram profiles engaged in coordinated inauthentic behavior. The DFRLab discovered that some of these accounts used AI-generated images and stock photo imagery as their profile photo. While the research did not find instances regarding who could be behind such an operation or the purpose of the network, it found connections between the assets and the Argentine political movement Kirchnerism.

Some of the user accounts declared in their descriptions their support for the Argentine political movement Peronism and its successor, Kirchnerism, while others shared content in favor of Sergio Berni, the minister of security for the province of Buenos Aires, Argentina’s capital.

Although the accounts did not post much content to their timelines, the DFRLab was able to corroborate the connections between the assets, as many of them followed each other on Instagram and shared friendship connections on Facebook.

In its announcement of the takedown, Facebook stated:

The people behind this activity used fake accounts — some of which had already been detected and removed by our automated systems — to create elaborate fictitious personas using profile photos generated by artificial intelligence, post, like and comment on content to make it appear more popular than it was. This network posted in Spanish and focused primarily on inauthentically amplifying posts and news articles related to Sergio Berni, Buenos Aires’ Minister of Security. These accounts liked and re-shared the same posts at once from this politician’s official Page. They also posted about hobbies, animals, and cooking to appear more authentic.

We found this network after reviewing public information about a small portion of this activity shared by an open source researcher. Our assessment benefited from additional findings shared with us by FireEye, a cybersecurity company.

Network’s links to Kirchnerism

The accounts in this network posed as locals, supposedly based in Buenos Aires. The DFRLab reviewed public posts on 662 Facebook accounts and 226 Instagram accounts that were removed from the platform. According to the accounts’ descriptions on Facebook, some of the assets supported the Argentine political movement Peronism, supporting the legacy of Juan Domingo Perón, who served as president of Argentina between 1946 and 1955 and then between 1973 and 1974. Some accounts also included descriptions supporting Kirchnerism, another political movement based on the ideals of presidents Néstor Kirchner and Cristina Fernández de Kirchner, who governed Argentina from 2003 to 2015. Fernández de Kirchner is currently the vice president in Alberto Fernández government.

Screengrabs from some of the removed Facebook accounts containing sentences in their descriptions supporting both Peronism and Kirchnerism (as underlined). (Source: Facebook)

The research also found connections between the removed assets and Kirchnerism by analyzing the pages liked by the accounts. On Facebook, some of these pages were connected primarily to current Argentinian President Fernández, current Vice President Fernández de Kirchner, and two local politicians linked to Kirchnerism: Axel Kicillof, the governor of Buenos Aires, and Sergio Berni, the minister of security for Buenos Aires.

The accounts also liked assets connected to political parties, movements, and organizations including the aforementioned Peronism and Kirchnerism, as well as the left-wing coalition Frente De Todos (Everybody’s Front), which supported Alberto Fernández in the Argentine November 2019 presidential elections.

Screengrabs show Facebook pages liked by two of the analyzed assets involved in the takedown. Some of these pages were connected to Argentina’s current President Alberto Fernández (orange boxes), former President Cristina Fernández de Kirchner (green boxes), Buenos Aires’ governor Axel Kicillof (purple boxes), and Sergio Berni, Minister of Security in Buenos Aires, Argentina (yellow box). The accounts also liked assets connected to political parties, movements, and organizations such as Frente De Todos (blue boxes), peronismo (red boxes), and kirchnerismo (black box). (Source: Facebook)

The network’s activity on Facebook showed signs of coordination. Many assets had their last post between May and August 2020. On some days, around 50 Facebook accounts from the network shared or updated their profile on the same day.

Graph showing the last activity of the analyzed assets. Most of the Facebook accounts registered their last activity between May and August 2020. (Source: @estebanpdl/DFRLab)

Some of them updated their profile picture within the same hour.

Screengrabs from four accounts involved in the takedown that updated their profile photo on December 26, 2018. Two of these accounts (green boxes) updated their pictures within the same hour. (Source: Facebook)

The accounts did not share a lot of content on Facebook. Some accounts only had their profile and banner pictures as the last activity. However, a subset of the accounts shared similar posts, mostly highlighting the assistance provided by the Buenos Aires government to one of the neighborhoods in the province of Buenos Aires during the 2020 pandemic. The activity of the network’s accounts increased significantly after the first post on May 26, 2020.

Image showing one post from the removed account Belen Trotta shared on May 26, 2020, in which the operator lauded the Buenos Aires government for its efforts during the 2020 pandemic. Many of the assets involved in the takedown shared the same content between May and June 2020. (Source: @estebanpdl/DFRLab via Facebook)

Other accounts followed a similar pattern but with a different post related to Berni.

Image showing examples of Berni-related posts shared by the removed accounts on Facebook. (Source: Facebook)

On Instagram, the removed profiles showed connections with Kirchnerism as well, but to a lesser extent. A social network graph showed the abovementioned Argentine politicians among the most followed accounts by the analyzed assets.

A social network graph showing the accounts followed by the Instagram profiles removed by Facebook. (Source: @estebanpdl/DFRLab via Facebook)

By analyzing the Instagram connections, the DFRLab found that many of the removed assets also followed the account @madison.seguidores, which belonged to Buenos Aires-based digital marketing firm Madison. According to Madison’s website, the firm offers services to boost engagement and followers on Instagram for its clients. The DFRLab was unable to establish a connection between the Argentine politicians mentioned above and the firm.

Screengrabs from the Instagram account @madison.seguidores and Madison’s website. As a marketing company, Madison offers its clients increased engagement and followers on Instagram. (Source: @estebanpdl/DFRLab via Instagram, left; Madison/archive, bottom left; Madison/archive, right)

Analyzing the fake profiles

A large proportion of the Facebook and Instagram accounts had AI-generated profile pictures. They used generative adversarial networks (GANs), an artificial intelligence technology behind websites such as This Person Does Not Exist. The use of this technology in this network aimed to create fictitious personas to engage on the platforms, mainly to amplify Berni-related posts or to like pages linked to Kirchnerism.

Image showing eight examples of assets involved in the takedown that used AI-generated profile pictures. (Source: Facebook)

AI-generated faces using GANs are evolving toward more believable pictures; still, these profiles can be recognized by analyzing features on the image, such as background details and glitches in hair, lips, eyes, and ears.

Another feature of these AI-generated faces is that the eyes occur in the same place, regardless of the orientation of the computer-generated face. Detecting the eye-orientation in a collection of potential AI-generated images can be a useful technique to recognize these profiles on social media. The following image shows an example of this technique.

Image showing AI-generated faces used by the network. The yellow lines show how the eyes of the faces align both vertically and horizontally. (Source: @estebanpdl/DFRLab via Facebook)

Hundreds of assets involved in the takedown used AI-generated faces. The following images showed more than 80 AI profile pictures used in the network. In all these profiles, the eyes match in both vertical and horizontal aligns.

Image showing 84 AI-generated faces used by the network. The yellow lines show how the eyeballs of the faces match in both vertical and horizontal aligns. (Source: @estebanpdl/DFRLab via Facebook)

Another noteworthy technique to analyze these profiles, exposed by open source analyst @conspirator0 while analyzing thousands of AI-generated images from thispersondoesnotexist.com, is that the eye alignment can also be detected when faces are superimposed on one another.

The following graphic in .gif format shows a superimposed image of 83 AI-generated profiles involved in the takedown. The comparison shows how the eyes aligned precisely.

The graphic shows a superimposed image of 83 AI-generated profiles involved in the takedown. (Source: @estebanpdl/DFRLab via Facebook)

AI-generated faces were not the only method used by the network to create fictitious personas. Other accounts took their profile picture from stock photo imagery and websites. The following image shows examples of these profiles.

Image showing examples of some removed accounts that used profile pictures from image databases and websites. (Source: Facebook; Google)

Asset connections

The DFRLab mapped the connections between the assets on both Facebook and Instagram. Social network analysis techniques showed that the Facebook accounts shared friends on the platform, while the profiles on Instagram mainly followed each other.

Graph showing the friendship connections on Facebook. The nodes in orange represent the analyzed user accounts. (Source: @estebanpdl/DFRLab via Facebook)

On Instagram, the follower connections indicate that most of the removed profiles were following each other.

A social graph showing the follower connections between the Instagram removed assets. (Source: @estebanpdl/DFRLab via Facebook)

Conclusion

The accounts of this network did not show to be frequently active — at least not publicly. Nevertheless, the network’s connections to the Argentine political movement Kirchnerism suggested coordination of the user accounts, either through pages’ likes or similar posts supporting Kirchnerism-allied politician Sergio Berni.

Although profile photos of the accounts followed a particular pattern of taking images from elsewhere online, hundreds used AI-generated profile photos, which could have made it more difficult for social media users to detect the inauthenticity of the network.

As AI-generated faces evolve toward more realistic human faces, the persons behind social media-based information operations choose to create fictitious personas using such AI technology. For social media users, this represents a potential threat while engaging in political and democracy-related conversations.


Esteban Ponce de León is a Research Assistant, Latin America, with the Digital Forensic Research Lab.

Daniel Suárez Pérez is a Research Assistant, Latin America, with the Digital Forensic Research Lab.

Follow along for more in-depth analysis from our #DigitalSherlocks.