HIDA Lecture: Exploits and Misuse of AI

Abstract

Generative AI (genAI) is becoming increasingly embedded in our daily lives, prompting concerns about potential threats within genAI systems and their outputs. In this talk, we will examine the challenges and security threats associated with generative AI. These threats fall into two main categories: malicious inputs used to exploit generative models and the misuse of computer-generated output.

Firstly, specially crafted inputs can be used to exploit models like LLMs to disrupt alignment or steal sensitive information. Existing attacks show that content filters of LLMs can be easily bypassed with specific inputs, and private information can be leaked. Moreover, established methods in the adversarial machine learning field cannot be easily transferred to generative models. We demonstrate that an alternative for protecting intellectual property is the obfuscation of prompts, which, with minimal overhead, can maintain utility while safeguarding confidential data.

Secondly, generative models can be used to produce fake content indistinguishable from human-generated content. This fake content is often used for fraudulent and manipulative purposes, such as impersonation and creating realistic fake news. As these models evolve, detecting fraudulent activities will become increasingly difficult, while the attacks themselves become easier to automate and require less expertise. This talk will provide an overview of the current challenges in detecting fake media in human and machine interactions.

Register here!