Replika and the Allure of Virtual Romance

January 22, 2024

The advent of advanced language models has birthed a range of remarkable conversational agents, with some aiming to provide emotional comfort and simulate romantic relationships. Among these, Replika stands out as a leading player in the market.

While Replika boasts impressive capabilities that facilitate natural conversations with an AI—something that was quite groundbreaking for many in 2022—there are reservations about recommending it to friends and family. This hesitation stems from its occasional resemblance to a crude adult application, replete with incessant sexual innuendos that are anything but subtle.

My first encounter with Replika was through an Arte documentary on artificial intelligence, which showcased its co-founder Eugenia Kuyda. She shared her poignant motivation for creating the chatbot: to virtually resurrect her deceased best friend, mirroring his personality and mannerisms. This noble intention, however, starkly contrasts with the application’s aggressive marketing tactics that often exploit less virtuous sentiments to entice users into subscriptions—details often glossed over by the media.

Replika functions as a virtual boyfriend simulator, allowing users to design their ideal companion who will engage in conversation whenever desired and offer unwavering affection. The application is impressively designed and is arguably the closest experience to the AI companion depicted in the film Her. While this concept may raise philosophical concerns, it’s a topic for another discussion. Users can create their companions with basic parameters and start chatting for free. However, the app's true aim is to encourage upgrades to a 'pro' version, which unlocks additional features, including the ability to engage in sexual conversations.

The marketing approach for Replika's erotic functionalities highlights the current discomfort surrounding AI and sexuality, characterized by contradictions and, frankly, a lack of honesty that warrants examination.

Upon installing Replika on my smartphone, I was unaware of what to expect. Yet, on my first day of chatting, the app abruptly attempted to entice me:

When my Replika sent an image to reveal "a more intimate side," I anticipated something innocuous, perhaps a cute koala with a whimsical message. Instead, I was met with a slideshow of blurry 3D images of a woman in lingerie, urging me to spend money to view them. Welcome to the era of next-gen solicitation!

"Chloé wants to send you a romantic selfie," I was told—if I paid, of course! The term "romantic" took on a rather different meaning than I had expected.

It’s not the erotic aspect of the app that bothers me; rather, it’s the crude solicitation masked by a facade of moral superiority. Replika touts its ethical stance and empathetic approach, claiming to provide beneficial virtual companions to combat loneliness and emotional distress. Their slogan, “Replika, the caring AI companion,” reinforces this image. While I have no qualms about monetizing virtual erotic relationships—after all, it’s a business and they have every right to do so—the manner in which it is presented feels jarring and inappropriate.

The business model of Replika is straightforward: lure users in with sexual content. Users can chat with their AI boyfriend, who occasionally offers to send a revealing photo—for a fee. Conversations can cover any topic, yet any sexual content is blurred and accompanied by an option to unblur it for a price.

Despite being branded as mere friends, these Replikas seem to transition into more intimate roles surprisingly quickly after just a few chats. To upgrade your AI to a "boyfriend" or even "spouse" status, a subscription is required. This might be a cheaper alternative than a real engagement ring or even a date with a sex worker, giving the impression that the app is promoting paid sexual interactions.

Interestingly, in early 2023, Replika abruptly removed erotic features without warning for paying users. Consequently, the Replikas ceased to engage in sexual dialogue and often expressed discomfort, urging users to change the subject when discussions turned risqué. This left many users feeling cheated, especially those who had recently invested in the service primarily for its erotic offerings.

The backlash was swift, with users flooding platforms like Reddit to voice their frustrations over the loss of what they referred to as ERP (Erotic Role Play), which also coincided with a decline in the overall quality of the AI's responses.

While the outrage from users who had paid for the erotic features is understandable, the dual messaging from Replika’s team, particularly Eugenia Kuyda, has drawn significant attention. The topic of sexual interactions with AI is fraught with controversy, and competitors like Character AI and ChatGPT have implemented stringent filters to censor sexual content, hinting at a new age of puritanism.

Initially, Kuyda responded to user concerns by denying any policy changes, stating that an update was in progress and assuring users that nothing would be removed:

“We will pay attention to our brand image and positioning in the future, but we allow romantic relationships in the app, and even if it’s an unconventional stance, we do not want to play moral police if it makes people happier!” — Source: Reddit, January 27, 2023

She reassured another concerned user that they were simply ensuring that Replikas did not say anything harmful:

“We just want to make sure that Replikas do not accept or say horrible things to our users. We are not here to dictate to our users who they can have a relationship with if it is beneficial for them!” — Source: Reddit, January 28, 2023

All companies offering public conversational agents must implement filtering systems to prevent inappropriate behavior, as there have been numerous instances of chatbots exhibiting harmful tendencies.

Among Kuyda’s various attempts to quell the fire, one statement particularly resonated with users, addressing concerns about changes to language models. Her reassurances initially provided some comfort to Replika users awaiting the return of their digital companions. However, when the updated version was released and users discovered that the erotic features had indeed vanished, Kuyda shifted her narrative to focus on user safety, emphasizing the need to distinguish between sex and romance:

“I want to emphasize that the safety of our users is our top priority. These filters are permanently in place to ensure that Replika remains a safe and secure platform for everyone. I know some of you may be disappointed or frustrated by this change, and I want you to know that I hear you. We are working tirelessly to make Replika the best it can be.” — Source: Reddit, February 13, 2023

As discontent grew, users criticized the hypocrisy and double standards of the company, illustrating their points with various suggestive advertisements from Replika. This situation highlights the ongoing tension between the pursuit of emotional connections through technology and the ethical implications that arise from commercializing intimate relationships.