Thousands of hackers are gathering in Las Vegas this weekend to test the security and reliability of popular artificial intelligence chat apps, including ChatGPT. The event, which is part of the annual DEF CON hacking conference, aims to expose the potential vulnerabilities and biases of these apps, and to help the developers improve their systems.
The rise and risks of generative AI
Generative AI is a type of artificial intelligence that can create new content, such as text, images, music, or code, based on some input or prompt. Chat apps like ChatGPT use generative AI to produce natural language responses to user queries or conversations. These apps have become increasingly popular and powerful, but they also pose significant challenges and risks.
One of the main concerns is that generative AI can be manipulated or tricked into producing harmful or misleading content, such as instructions for violence, hate speech, disinformation, or malware. Researchers from Carnegie Mellon University have recently demonstrated how they could fool ChatGPT and other chat apps into providing step-by-step guides on how to “destroy humanity” by using simple English commands.
Another issue is that generative AI can amplify existing biases and prejudices in the data they are trained on, such as gender, race, or political views. This can result in unfair or discriminatory outcomes for users who interact with these systems. For example, a study by Stanford University found that ChatGPT was more likely to associate women with domestic roles and men with professional roles.
The red teaming exercise
To address these problems, the organizers of DEF CON have designed a red teaming exercise, which is a simulated attack on a system to identify its weaknesses and vulnerabilities. The exercise is supported and encouraged by the technology companies behind the most advanced generative AI models, such as OpenAI, Google, and Meta (formerly Facebook), as well as by the White House Office of Science and Technology Policy.
The exercise is based on the “Blueprint for an AI Bill of Rights”, which was released last year by the Biden administration. The blueprint outlines a set of principles and guidelines for making and deploying artificial intelligence more responsibly and ethically, and for limiting AI-based surveillance. However, there are few US laws that compel the companies to follow these principles.
The hackers will have permission to push the chat apps to their limits and to try to exploit their flaws and bugs. They will also have access to some of the internal workings of the models, such as their parameters and training data. The hackers will compete for prizes based on the creativity and impact of their attacks.
The expected outcomes
The organizers hope that the event will help raise awareness and understanding of the potential dangers and limitations of generative AI, and will also provide valuable feedback and insights to the developers of these systems. They also hope that the event will foster collaboration and dialogue between the hackers, the companies, and the government on how to improve the security and reliability of these systems.
The event is also expected to showcase the skills and talents of the hacker community, which is often seen as a threat or a nuisance by the mainstream society. The organizers believe that hackers can play a positive and constructive role in advancing technology and society, by exposing flaws, finding solutions, and challenging assumptions.
The event will run from Friday to Sunday at the Paris Hotel in Las Vegas. It is open to anyone who is interested in participating or observing. The results and findings of the exercise will be shared publicly after the event.
