netgoat-ai/Koda-IDS-Frontier-GRPO-Modes

Koda IDS Frontier — GRPO modes (stage 6 + stage 7)

Mode-tagged GRPO data for the Koda IDS Frontier 2.6B GPU pipeline and the 8B-A1B adapter produced by the TPU pipeline.

Both objectives are mode-aware: every scenario carries a reasoning_mode, a mode-composed scenario.system_context, and a mode-appropriate reference candidate. Prompts are not pre-rendered here; render them at train time with:

prompt = tokenizer.apply_chat_template(
    messages,
    tokenize=False,
    add_generation_prompt=True,
)

The template opens <think>. A none completion must close it immediately (</think>\n\nanswer); higher modes reason inside it and close it before the answer.

Files and mode counts

config	file	rows	none	low	med	high	xhigh	ultron
stage7	grpo_stage7_modes_246.parquet	246	23	57	76	63	17	10
stage6	grpo_stage6_modes_42.parquet	42	2	7	12	12	6	3

The same rows are also published as JSONL (grpo_stage7_modes_246.jsonl, grpo_stage6_modes_42.jsonl).

stage7 is the combined failure-targeting/contrastive set: 150 failure-targeting rows + 96 contrastive rows. The per-file artifacts were transformed with --stage grpo7; their mode counts sum exactly to the 246-row combined artifact (verified by ID and by mode).

Schema

Stage 6 and stage 7 share the nested scenario shape:

{
  "id": "KODA-GRPO-000001",
  "category": "network",
  "difficulty": "medium",
  "reasoning_mode": "high",
  "scenario": {
    "system_context": "<canonical prompt + mode directive>",
    "user_message": "<scenario>",
    "known_facts": ["..."],
    "unknowns": ["..."],
    "available_tools": ["..."],
    "reasoning_mode": "high"
  },
  "candidates": {
    "A": {"answer": "...", "think": ["..."], "reward_criteria": "..."},
    "B": {"answer": "...", "think": ["..."], "reward_criteria": "..."},
    "C": {"answer": "...", "think": ["..."], "reward_criteria": "..."},
    "D": {"answer": "...", "think": ["..."], "reward_criteria": "..."}
  },
  "evaluation": {"winner": "C", "rationale": "..."},
  "reward_criteria": {"..." : "..."}
}

The selected candidate is a reference, not a target to pad. Reference traces are trimmed to the requested mode ceiling when too long and are never expanded to meet a mode quota. A short reference is left short; the reward scores generated completions, not the reference candidate.

Objectives

• Stage 6 (42 rows): reasoning-quality GRPO on the curated netgoat-ai/MiMo-V2.6-Flash-Reasoning-GRPO pool. Contrastive candidates teach evidence calibration and uncertainty.
• Stage 7 (246 rows): failure-targeting GRPO. 150 rows target specific failure modes (premature containment, fabricated telemetry, severity inflation, defensive over-refusal); 96 rows are contrastive pairs. The combined set is what the GPU notebook loads.

The reward is v19: task_completion, evidence_grounding, technical_correctness, reasoning_quality, uncertainty_calibration, investigation_quality, hypothesis_quality, action_proportionality, tool_correctness, reasoning_efficiency, answer_shape, and mode_compliance. Actionable offensive guidance is penalised; naming a technique in a defensive report is not.

Harm filtering

The curated VSA/GRPO rows are not run through the external-corpus harm heuristic. That heuristic false-positives on defensive hypotheses such as “the attacker may have used an interactive shell”. Harm filtering is applied only to the external Trendyol replacement corpus.

Usage

from datasets import load_dataset

stage7 = load_dataset(
    "netgoat-ai/Koda-IDS-Frontier-GRPO-Modes",
    "stage7",
    split="train",
)
row = stage7[0]
print(row["reasoning_mode"], row["evaluation"]["winner"])

For TRL GRPO, build the chat messages from scenario.system_context and scenario.user_message, render with add_generation_prompt=True, and pass reasoning_mode to the reward function. Do not append a separate <think> opening tag; the chat template already provides it.

Validation

• 42/42 stage-6 rows and 246/246 stage-7 rows have one mode directive in scenario.system_context.
• Stage 7 IDs are unique; the 150+96 per-file set equals the combined 246-row set exactly (IDs, candidates, evaluation, scenario, mode, meta).
• Every none row's selected reference has an empty think list.
• No legacy reasoning_depth field remains; the mode is the depth contract.
• Reference candidates are not padded; the transformed files only trim over-budget traces.

Attribution and license

Curated Koda GRPO data, Apache-2.0. Built for defensive cybersecurity investigation research. No offensive operational guidance is intended.