Text-to-Image AI Developed Its Own 'Language', Study Claims

CC0 / / Artificial intelligence
Artificial intelligence - Sputnik International, 1920, 04.06.2022
The discovery of a “language” used by AI presents “many interesting security and interpretability challenges”, one of the authors of a new study suggests.
DALL-E 2, an artificial intelligence program that produces images from textual descriptions, may have had created a very “hidden vocabulary”, computer science PhD student Giannis Daras and co-author Alexandros Dimakis argue in a new study.
Offering a brief overview of their findings in a Twitter thread, Daras noted how DALL-E 2 has a known limitation as it “struggles with text”: for example, the prompt "Two farmers talking about vegetables, with subtitles" results in a picture that displays two farmers talking, but the subtitles appear to be complete gibberish.
“However, the text is not as random as it initially appears”, Daras points out. We feed the text "Vicootes" from the previous image to DALLE-2. Surprisingly, we get (dishes with) vegetables! We then feed the words: "Apoploe vesrreaitars" and we get birds. It seems that the farmers are talking about birds, messing with their vegetables!”
Daras also remarked that some words from the DALL-E 2 “language” can be used to create “absurd prompts”, and that the discovery of the AI’s “language” presents “many interesting security and interpretability challenges”.
“Currently, NLP systems filter text prompts that violate the policy rules”, the study’s author elaborated. “Gibberish prompts may be used to bypass these filters”.
To participate in the discussion
log in or register
Заголовок открываемого материала