As the one-year anniversary of ChatGPT approaches, the cybersecurity landscape continues to evolve, with both defensive teams and threat actors exploring new possibilities offered by generative AI, particularly large language models (LLMs). While LLMs have the potential to level the playing field for cybersecurity analysts, there are growing concerns about how threat actors can harness these technologies to their advantage. This article sheds light on a prominent concern in the cybersecurity community: the use of prompt injections to manipulate ChatGPT and other generative AI tools, potentially leading to data breaches and social engineering attacks.
The expanding attack landscape
Generative AI tools, like ChatGPT, have expanded the attack landscape in the cybersecurity domain. Their versatile use and accessibility have created new opportunities for both accidental data leaks and malicious exploitation by threat actors. Unlike traditional security systems, generative AI relies on the data provided by users, which introduces a level of unpredictability and vulnerability. Threat actors recognize these vulnerabilities and view tools like ChatGPT as a means to craft more convincing and targeted social engineering attacks.
The power of prompt injections
One of the tactics that threat actors employ to manipulate ChatGPT is prompt injections. Prompt injections involve crafting deceptive and manipulative language within a prompt, causing the AI to generate responses that may bypass safety measures or produce malicious outputs. In essence, prompt injections can be likened to SQL injections in the cybersecurity realm, as they involve manipulating the system by exploiting seemingly normal directives.
GitHub explains prompt injection as “a type of security vulnerability that can be exploited to control the behavior of a ChatGPT instance.” This means that a simple prompt injection can instruct the LLM to ignore pre-programmed instructions, perform nefarious actions, or circumvent filters to generate incorrect or harmful responses.
The risk to sensitive data
Generative AI heavily relies on user-generated data sets, and users often provide increasingly sensitive information to elicit specific responses. This practice can inadvertently put sensitive data at risk. When threat actors employ prompt injections, they may strategically engineer prompts to gain access to sensitive information, employing social engineering tactics to extract valuable content.
Sensitive information, such as proprietary strategies, product details, or customer information, may be at stake. If a maliciously engineered prompt is successfully executed, threat actors could potentially access this information. Additionally, prompt injections could lead users to malicious websites or exploit vulnerabilities within systems.
Protecting your data
In light of these emerging threats, it is imperative to adopt best security practices when utilizing LLM models like ChatGPT. Here are some steps to consider:
Exercise Caution with Sensitive Information: Avoid sharing sensitive or proprietary data in generative AI tools whenever possible. If it is essential for such information to be accessible for task completion, ensure that it is anonymized and presented in a generic form to minimize risks.
Verify Before Trusting: Before complying with any request generated by generative AI, conduct due diligence to verify its legitimacy. Whether it’s responding to an email or visiting a website, always verify the path’s authenticity.
Report Suspicious Activity: If something appears suspicious or out of the ordinary, do not hesitate to contact your IT and security teams for assistance. Early reporting can be crucial in mitigating potential threats.
