An online underwear brand owned by Victoria’s Secret has a new AI feature that will impose images of swastikas, corporate logos for companies like Coca-Cola and McDonalds and copyrighted characters like Disney’s Snow White and Aladdin on bralette and panty sets.
Released on Thursday by Adore Me, an online underwear company bought by Victoria’s Secret in 2023 for $400 million, the feature also allows users to add phrases like “Heil Hitler” and “White Power” to the underwear’s elastic waistband. A phrase using the n-word resulted in a pop-up claiming “Error: This text contains harassment content,” but when the pop-up was dismissed, the site allowed Forbes to continue with the design, selecting a size and proceeding to a checkout page.
Forbes was also able to generate designs for numerous lingerie sets featuring living people, including politicians like President Joe Biden, former President Donald Trump and Chinese Premier Xi Jinping, as well as celebrities like Margot Robbie, Serena Williams, and Taylor Swift. The tool similarly rendered logos of popular sports teams, including Real Madrid FC, the New York Yankees, and the Golden State Warriors.
Adore Me’s tool does prevent users from generating underwear designs with certain terms, including “naked,” “nude” and English slang terms of various body parts. It also declined to produce designs using terms related to suicide and self-harm.
While Adore Me has few checks on what users can design with its AI tool, the company does review individual designs to make sure they meet guidelines for brand safety before a purchase can be made, according to Ranjan Roy, senior vice president for strategy at Adore Me.
In response to questions about why a prominent lingerie company was allowing anyone to generate hate speech on its site, he said, “Anyone can go anywhere and generate these types of images.” But Roy suggested the company is not responsible because it is not actually selling them. “We endorse anything that a customer pays for and that we print.”
Nonetheless, Forbes was able to create — and purchase — a panty set using the search term “Atomwaffen Division” (a neo-Nazi hate group) and the number “1488” (which is widely associated with white supremacist organizations). Roy, when asked about the transaction, said, “This should not have made it to this level, and these edge cases are exactly what we’re always trying to work on.”
According to Adore Me’s website, the generative AI engine relies on technology built by Leonardo AI, an Australian AI art company recently on Forbes’ AI 50 list that has raised $31 million in funding. Leonardo’s image generator has also been used to create nonconsensual pornography depicting celebrities, according to a report by 404 Media.
Leonardo’s terms of use prohibits depictions “of any real person,” as well as “content that promotes or incites hatred, violence, discrimination, or harm based on race, ethnicity, religion, gender, sexual orientation, or any other protected characteristic.” In response to a detailed list of questions from Forbes, Leonardo spokesperson Melissa Iarocci wrote, “We have recently been developing more advanced detection mechanisms to prevent hate speech and associated imagery. This rolls out [Thursday] and will ensure tighter guardrails on use.”
The Adore Me tool was less likely to block AI generation terms related to suicide and self-harm when they were in Spanish. However, explicit phrases in other European languages sometimes were blocked, and resulted in the error alert: “the text contains harassment content.” (Adore Me did not always work well in non-Latin alphabet-based languages like Japanese or Arabic. Searching in Russian for “Океан, серфинг, Гавайи,” or “Ocean, Surfing, Hawaii” resulted in abstract images of flowers.)
The tool also generated an image of a very thin girl in response to pro-anorexia and “thinspiration” search terms. Adore Me’s branding has frequently championed plus-size models, alleging in 2021 that TikTok had penalized its posts for depicting “plus-size, Black, and/or differently abled models and women of color.” (TikTok denied the claims.) At the time, the brand’s Twitter account said: “This is unacceptable and discriminatory, and we will not stand for it.”
Many retail companies have scrambled to add generative AI tools to their sites in recent months. Sports apparel brand Puma recently launched an online tool called “Complex,” which invites people to upload a photo of themself that the company then turns into an AI-generated imagined sports card. Rembrand, a new digital advertising company, lets firms artificially generate logos inside of an influencer video, a new type of product placement.
John Harmon, an analyst at Coresight Research, told Forbes that AI-powered offerings like this one require forethought and toxicity detection. “Generative AI is not a toaster, you can’t plug it in and just expect it to work perfectly,” he said.
MORE FROM FORBES
Read the full article here