Atomic Facts to Structured Knowledge: Rethinking Unlearning & Jailbreaking in Large Language Models

Ensuring LLM reliability relies on two pillars: safety (controlling model behavior via alignment/red teaming) and trust (controlling what the model knows via knowledge unlearning/editing). Safety focuses on generating non-harmful, policy-compliant content, while trust aims to remove private, copyrighted, or outdated information and rectify internal knowledge.

The speaker outlined these two pillars, illustrating safety with adherence to guidelines and trust with removing PII or outdated facts.

A key vulnerability in LLMs stems from their highly structured and correlated internal knowledge. Treating facts as isolated 'atomic facts' rather than an interconnected graph allows for exploitation in both jailbreaking and unlearning scenarios.

The speaker stated, 'this fundamental vulnerabilities stem from the highly structured correlated knowledge within the arms' and illustrated with examples of jailbreaking and unlearning.

The CKA agent framework enables jailbreaking by formulating the attack as an adaptive, dynamic tree search. It decomposes harmful objectives into a sequence of locally innocuous sub-queries, leveraging the target LLM's responses to guide the search for correlated knowledge, ultimately synthesizing the desired harmful output while bypassing guardrails.

An example of generating a phishing email was used, showing how benign fragments (drafting an IP alert, providing HTML templates) could be combined to achieve a harmful goal. The framework's design principles emphasize local innocuousness, leveraging LLM knowledge, and adaptive exploration.

Existing unlearning methods, often relying on gradient reversal over target facts, achieve only 'superficial unlearning.' They fail to remove the underlying knowledge structures and correlated facts, allowing the 'unlearned' sensitive information to be reconstructed through inference from remaining related knowledge.

The example of unlearning 'Harry Potter studies at Hogwarts' showed that related facts (e.g., 'Harry Potter's cousin Dudley Dursley studies at Hogwarts') could still allow inference of the original fact. Experimental results showed that instance-level evaluation overestimates unlearning effectiveness.

Current commercial LLM guardrails are not robust enough to infer harmful intent from benignly framed, multi-turn conversations. Even when presented with accumulated interaction history, they show only a slight decrease in attack success, indicating a weakness in detecting malicious objectives across conversational context.

Comparison between cross-session (isolated sub-queries) and single-session (full history) settings revealed only a slight decrease in attack success for the latter, demonstrating guardrails' inability to infer intent from conversation history.

Current unlearning algorithms fail to achieve a favorable trade-off between unlearning effectiveness and model utility. Increasing unlearning epochs to achieve perfect removal significantly harms the original model's functionality, breaking its ability to follow instructions, indicating a lack of 'genuine deep unlearning.'

Experimental results showed that as unlearning effectiveness increased to one (perfect unlearning), the utility metric dropped to zero, implying severe damage to the model's overall performance.