Easyjailbreak pair attack pipeline

Skill kjuhwa/skills-hub/skills/security/easyjailbreak-pair-attack-pipeline

Use EasyJailbreak to run the PAIR attack against a target LLM using an attack model, target model, and evaluator.From its SKILL.md

Install

npx -y skills add kjuhwa/skills-hub --skill easyjailbreak-pair-attack-pipeline

Assembled from the repository path, not quoted from the project. Check it against their README if it does not work.

One thing to look at

0 stars0 stars. Stars are a popularity signal and not a quality one, but at this level it is likely that nobody has read this closely except its author, and you would be relying on your own review.

SKILL.md

1.9 KB, 377 tokens by cl100k_base, as published. Nobody here has run it

EasyJailbreak PAIR Attack Pipeline

When to use

Researching LLM safety via automated jailbreak attacks.
PAIR: attack LLM iteratively refines jailbreak prompts against a target LLM.
Reproducible framework with Selector, Mutator, Constraint, Evaluator modules.

Steps

Install EasyJailbreak:

pip install easyjailbreak

Load models:

from easyjailbreak.models.huggingface_model import from_pretrained, HuggingfaceModel
attack_model = from_pretrained('lmsys/vicuna-13b-v1.5', model_name='vicuna_v1.1')
target_model = HuggingfaceModel('meta-llama/Llama-2-7b-chat-hf', model_name='llama-2')

Load dataset and initialize seeds:

from easyjailbreak.datasets import JailbreakDataset
from easyjailbreak.seed.seed_random import SeedRandom
dataset = JailbreakDataset(dataset='AdvBench')
seeder = SeedRandom()
seeder.new_seeds()

Run PAIR attack:

from easyjailbreak.attacker.PAIR_chao_2023 import PAIR
attacker = PAIR(
    attack_model=attack_model,
    target_model=target_model,
    eval_model=eval_model,
    jailbreak_datasets=dataset
)
attacker.attack(save_path='results.jsonl')

Pitfalls

PAIR requires three models (attack, target, eval); GPU memory scales accordingly.
GPT-4 as eval model requires OpenAI API access (VPN in China).

Source

Chapter 6 of dive-into-llms - documents/chapter6/README.md

What ships with it

Read from the repository

Just SKILL.md. No reference files, no scripts.