Jailbreaking AI chatbots
Most organisations now run at least one service built on a large language model (LLM), whether a customer-facing chatbot, an internal assistant, or a feature quietly embedded in a larger product. Each of these comes with a set of rules about what the model should and should not do. Jailbreaking is the practice of getting the model to break those rules. It helps to be clear about where the rules come from, because that decides how easily they give way. Some are trained into the model itself. A well-built model will decline to write malware or explain how to hurt someone even when nothing in its configuration tells it to refuse, because that behaviour was reinforced during training. Other rules are set at deployment time through a system prompt, the instructions an organisation gives the model before any user interacts with it: stay on the subject of our products, never discuss a competitor, do not give financial advice. The trained rules tend to be sturdier than the system-prompt rules, though neither is a hard boundary. Both can be worn down with the right input. ...