How much do language models memorize?
Paper • 2505.24832 • Published • 4
Dataset Viewer
The dataset viewer is not available for this split.
Cannot load the dataset split (in streaming mode) to extract the first rows.
Error code: StreamingRowsError
Exception: CastError
Message: Couldn't cast
name: string
mode: string
n_params: int64
config: struct<test_size: int64, steps: int64, batch_size: int64, eval_batch: int64, eval_every: int64, lr: (... 123 chars omitted)
child 0, test_size: int64
child 1, steps: int64
child 2, batch_size: int64
child 3, eval_batch: int64
child 4, eval_every: int64
child 5, lr: double
child 6, seed: int64
child 7, project: string
child 8, space_id: string
child 9, n_layer: int64
child 10, d_model: int64
child 11, n_head: int64
child 12, dataset_size: int64
history: list<item: struct<train_bits_per_token: double, test_bits_per_token: double, memorized_bits_total: d (... 117 chars omitted)
child 0, item: struct<train_bits_per_token: double, test_bits_per_token: double, memorized_bits_total: double, memo (... 105 chars omitted)
child 0, train_bits_per_token: double
child 1, test_bits_per_token: double
child 2, memorized_bits_total: double
child 3, memorized_bits_per_param: double
child 4, dataset_bits: double
child 5, train_step: int64
child 6, lr: double
child 7, steps_per_sec: double
final: struct<train_bits_per_token: double, test_bits_per_token: double, memorized_bits_total: double, memo (... 51 chars omitted)
child 0, train_bits_per_token: double
child 1, test_bits_per_token: double
child 2, memorized_bits_total: double
child 3, memorized_bits_per_param: double
child 4, dataset_bits: double
paper_alpha_bf16_mean: double
paper_alpha_fp32_mean: double
mo
...
, memorized_bits: double, bits_per_param: double, fr (... 117 chars omitted)
child 0, item: struct<N: int64, dataset_bits: double, memorized_bits: double, bits_per_param: double, frac_dataset_ (... 105 chars omitted)
child 0, N: int64
child 1, dataset_bits: double
child 2, memorized_bits: double
child 3, bits_per_param: double
child 4, frac_dataset_memorized: double
child 5, train_bits_per_token: double
child 6, test_bits_per_token: double
child 7, rising_last_1k_steps: bool
child 2, L8d256 (6.86M): struct<n_params: int64, alpha_lower_bound_bpp: double, runs: list<item: struct<N: int64, dataset_bit (... 179 chars omitted)
child 0, n_params: int64
child 1, alpha_lower_bound_bpp: double
child 2, runs: list<item: struct<N: int64, dataset_bits: double, memorized_bits: double, bits_per_param: double, fr (... 117 chars omitted)
child 0, item: struct<N: int64, dataset_bits: double, memorized_bits: double, bits_per_param: double, frac_dataset_ (... 105 chars omitted)
child 0, N: int64
child 1, dataset_bits: double
child 2, memorized_bits: double
child 3, bits_per_param: double
child 4, frac_dataset_memorized: double
child 5, train_bits_per_token: double
child 6, test_bits_per_token: double
child 7, rising_last_1k_steps: bool
to
{'models': {'L2d64 (0.24M)': {'n_params': Value('int64'), 'alpha_lower_bound_bpp': Value('float64'), 'runs': List({'N': Value('int64'), 'dataset_bits': Value('float64'), 'memorized_bits': Value('float64'), 'bits_per_param': Value('float64'), 'frac_dataset_memorized': Value('float64'), 'train_bits_per_token': Value('float64'), 'test_bits_per_token': Value('float64'), 'rising_last_1k_steps': Value('bool')})}, 'L4d128 (1.06M)': {'n_params': Value('int64'), 'alpha_lower_bound_bpp': Value('float64'), 'runs': List({'N': Value('int64'), 'dataset_bits': Value('float64'), 'memorized_bits': Value('float64'), 'bits_per_param': Value('float64'), 'frac_dataset_memorized': Value('float64'), 'train_bits_per_token': Value('float64'), 'test_bits_per_token': Value('float64'), 'rising_last_1k_steps': Value('bool')})}, 'L8d256 (6.86M)': {'n_params': Value('int64'), 'alpha_lower_bound_bpp': Value('float64'), 'runs': List({'N': Value('int64'), 'dataset_bits': Value('float64'), 'memorized_bits': Value('float64'), 'bits_per_param': Value('float64'), 'frac_dataset_memorized': Value('float64'), 'train_bits_per_token': Value('float64'), 'test_bits_per_token': Value('float64'), 'rising_last_1k_steps': Value('bool')})}}, 'paper_alpha_bf16_mean': Value('float64'), 'paper_alpha_fp32_mean': Value('float64')}
because column names don't match
Traceback: Traceback (most recent call last):
File "/src/services/worker/src/worker/utils.py", line 147, in get_rows_or_raise
return get_rows(
dataset=dataset,
...<4 lines>...
column_names=column_names,
)
File "/src/libs/libcommon/src/libcommon/utils.py", line 272, in decorator
return func(*args, **kwargs)
File "/src/services/worker/src/worker/utils.py", line 127, in get_rows
rows_plus_one = list(itertools.islice(safe_iter(ds, dataset=dataset), rows_max_number + 1))
File "/src/services/worker/src/worker/utils.py", line 483, in safe_iter
yield from ds.decode(False) if ds.features else ds
File "/usr/local/lib/python3.14/site-packages/datasets/iterable_dataset.py", line 2840, in __iter__
for key, example in ex_iterable:
^^^^^^^^^^^
File "/usr/local/lib/python3.14/site-packages/datasets/iterable_dataset.py", line 2373, in __iter__
for key, pa_table in self._iter_arrow():
~~~~~~~~~~~~~~~~^^
File "/usr/local/lib/python3.14/site-packages/datasets/iterable_dataset.py", line 2398, in _iter_arrow
for key, pa_table in self.ex_iterable._iter_arrow():
~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^
File "/usr/local/lib/python3.14/site-packages/datasets/iterable_dataset.py", line 536, in _iter_arrow
for key, pa_table in iterator:
^^^^^^^^
File "/usr/local/lib/python3.14/site-packages/datasets/iterable_dataset.py", line 419, in _iter_arrow
for key, pa_table in self.generate_tables_fn(**gen_kwags):
~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^
File "/usr/local/lib/python3.14/site-packages/datasets/packaged_modules/json/json.py", line 343, in _generate_tables
self._cast_table(pa_table, json_field_paths=json_field_paths),
~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.14/site-packages/datasets/packaged_modules/json/json.py", line 132, in _cast_table
pa_table = table_cast(pa_table, self.info.features.arrow_schema)
File "/usr/local/lib/python3.14/site-packages/datasets/table.py", line 2378, in table_cast
return cast_table_to_schema(table, schema)
File "/usr/local/lib/python3.14/site-packages/datasets/table.py", line 2306, in cast_table_to_schema
raise CastError(
...<3 lines>...
)
datasets.table.CastError: Couldn't cast
name: string
mode: string
n_params: int64
config: struct<test_size: int64, steps: int64, batch_size: int64, eval_batch: int64, eval_every: int64, lr: (... 123 chars omitted)
child 0, test_size: int64
child 1, steps: int64
child 2, batch_size: int64
child 3, eval_batch: int64
child 4, eval_every: int64
child 5, lr: double
child 6, seed: int64
child 7, project: string
child 8, space_id: string
child 9, n_layer: int64
child 10, d_model: int64
child 11, n_head: int64
child 12, dataset_size: int64
history: list<item: struct<train_bits_per_token: double, test_bits_per_token: double, memorized_bits_total: d (... 117 chars omitted)
child 0, item: struct<train_bits_per_token: double, test_bits_per_token: double, memorized_bits_total: double, memo (... 105 chars omitted)
child 0, train_bits_per_token: double
child 1, test_bits_per_token: double
child 2, memorized_bits_total: double
child 3, memorized_bits_per_param: double
child 4, dataset_bits: double
child 5, train_step: int64
child 6, lr: double
child 7, steps_per_sec: double
final: struct<train_bits_per_token: double, test_bits_per_token: double, memorized_bits_total: double, memo (... 51 chars omitted)
child 0, train_bits_per_token: double
child 1, test_bits_per_token: double
child 2, memorized_bits_total: double
child 3, memorized_bits_per_param: double
child 4, dataset_bits: double
paper_alpha_bf16_mean: double
paper_alpha_fp32_mean: double
mo
...
, memorized_bits: double, bits_per_param: double, fr (... 117 chars omitted)
child 0, item: struct<N: int64, dataset_bits: double, memorized_bits: double, bits_per_param: double, frac_dataset_ (... 105 chars omitted)
child 0, N: int64
child 1, dataset_bits: double
child 2, memorized_bits: double
child 3, bits_per_param: double
child 4, frac_dataset_memorized: double
child 5, train_bits_per_token: double
child 6, test_bits_per_token: double
child 7, rising_last_1k_steps: bool
child 2, L8d256 (6.86M): struct<n_params: int64, alpha_lower_bound_bpp: double, runs: list<item: struct<N: int64, dataset_bit (... 179 chars omitted)
child 0, n_params: int64
child 1, alpha_lower_bound_bpp: double
child 2, runs: list<item: struct<N: int64, dataset_bits: double, memorized_bits: double, bits_per_param: double, fr (... 117 chars omitted)
child 0, item: struct<N: int64, dataset_bits: double, memorized_bits: double, bits_per_param: double, frac_dataset_ (... 105 chars omitted)
child 0, N: int64
child 1, dataset_bits: double
child 2, memorized_bits: double
child 3, bits_per_param: double
child 4, frac_dataset_memorized: double
child 5, train_bits_per_token: double
child 6, test_bits_per_token: double
child 7, rising_last_1k_steps: bool
to
{'models': {'L2d64 (0.24M)': {'n_params': Value('int64'), 'alpha_lower_bound_bpp': Value('float64'), 'runs': List({'N': Value('int64'), 'dataset_bits': Value('float64'), 'memorized_bits': Value('float64'), 'bits_per_param': Value('float64'), 'frac_dataset_memorized': Value('float64'), 'train_bits_per_token': Value('float64'), 'test_bits_per_token': Value('float64'), 'rising_last_1k_steps': Value('bool')})}, 'L4d128 (1.06M)': {'n_params': Value('int64'), 'alpha_lower_bound_bpp': Value('float64'), 'runs': List({'N': Value('int64'), 'dataset_bits': Value('float64'), 'memorized_bits': Value('float64'), 'bits_per_param': Value('float64'), 'frac_dataset_memorized': Value('float64'), 'train_bits_per_token': Value('float64'), 'test_bits_per_token': Value('float64'), 'rising_last_1k_steps': Value('bool')})}, 'L8d256 (6.86M)': {'n_params': Value('int64'), 'alpha_lower_bound_bpp': Value('float64'), 'runs': List({'N': Value('int64'), 'dataset_bits': Value('float64'), 'memorized_bits': Value('float64'), 'bits_per_param': Value('float64'), 'frac_dataset_memorized': Value('float64'), 'train_bits_per_token': Value('float64'), 'test_bits_per_token': Value('float64'), 'rising_last_1k_steps': Value('bool')})}}, 'paper_alpha_bf16_mean': Value('float64'), 'paper_alpha_fp32_mean': Value('float64')}
because column names don't matchNeed help to make the dataset viewer work? Make sure to review how to configure the dataset viewer, and open a discussion for direct support.
YAML Metadata Warning:empty or missing yaml metadata in repo card
Check out the documentation for more information.
Reproduction: "How much do language models memorize?" (arXiv:2505.24832) — small scale
Reproduction of Morris et al. 2025 at 0.2M–6.9M parameter scale within a ~$7 compute budget.
Section 3 (capacity on uniform random data) and a mini Section 4 (FineWeb unintended
memorization + MIA). All raw JSONs in results/, figures in analysis/, live dashboards in
abidlabs/lm-memorization-repro-trackio.
What was run
- Sec 3: GPT-style transformers trained from scratch on uniform random tokens (vocab 2048, BOS + 64 tokens = exactly 704 bits/sequence), 5000 steps, batch 2048, AdamW lr 1e-3, bf16. Models: L2/d64 (235K), L4/d128 (1.06M), L8/d256 (6.86M). Dataset sizes N = 512 … 131072. Metric: memorized bits = 704·N − NLL2(x|θ̂).
- Sec 4 mini: 16M-param model (GPT-2 vocab), FineWeb
sample-10BT64-token windows (exact dedup), reference model trained on 262,144 windows; subset models trained on N ∈ {2048…262144} windows, 4000 steps. Unintended memorization = NLL2(θ_ref) − min(NLL2(θ_ref), NLL2(θ̂)); MIA = best-F1 over thresholds.
Confirmed (paper findings that reproduced)
- Memorization is bounded by parameters, not data. For every model, memorized bits grow ~linearly with dataset bits while below capacity, then flatten in absolute terms. Peak absolute memorization: 0.74M bits (235K params), 3.46M bits (1.06M), ≥20M bits (6.86M, still climbing at run end) — scaling with parameters as the paper claims.
- Below capacity, ~98% of dataset bits get memorized (train NLL → ~0.15 bits/token).
- Test NLL on held-out uniform data degrades far past the 11-bit uniform entropy once the model memorizes — pure memorization has zero generalization on random data.
- Sec-4 shape: unintended memorization per window collapses as N grows (409.7 → 2.35 bits/window from N=2048 to N=262144); raw-loss MIA is perfect (F1=1.0) in the fully-memorized regime and degrades once memorization disappears.
Quantitative deviations (expected at this scale)
- α lower bound: 3.16 / 3.26 / ≥2.99 bpp (L2/L4/L8) vs paper 3.51–3.65 (bf16). All three are lower bounds (still rising at 5000 steps; paper trained ~200× longer).
- Larger models in our budget memorize less per param (L8 < L4 < L2 at convergence-ish N) — consistent with slower convergence at scale, pointing toward the paper's α from below.
- At dataset ≫ capacity, memorized bits decline with N (fewer epochs) — our fixed 5000-step budget measures training-budget-limited memorization there, not the paper's converged plateau.
Caveats
- MIA best-F1 at large N is inflated by label imbalance (members ≫ non-members); raw-loss F1 is the reliable trend. The delta-score F1 increase with N should be read as an upper bound, not a confirmed effect.
- The reference model is undertrained (112M tokens vs Chinchilla-optimal ~330M for 16M params), so mem_U magnitudes are larger than the paper's oracle-based numbers.
Synthetic results
| L/d | params | N | dataset bits | memorized bits | bits/param | frac dataset |
|---|---|---|---|---|---|---|
| 2/64 | 235,328 | 512 | 360,448 | 355,657 | 1.511 | 0.99 |
| 2/64 | 235,328 | 1,024 | 720,896 | 592,133 | 2.516 | 0.82 |
| 2/64 | 235,328 | 2,048 | 1,441,792 | 645,843 | 2.744 | 0.45 |
| 2/64 | 235,328 | 4,096 | 2,883,584 | 690,982 | 2.936 | 0.24 |
| 2/64 | 235,328 | 8,192 | 5,767,168 | 717,466 | 3.049 | 0.12 |
| 2/64 | 235,328 | 16,384 | 11,534,336 | 743,274 | 3.158 | 0.06 |
| 2/64 | 235,328 | 32,768 | 23,068,672 | 732,996 | 3.115 | 0.03 |
| 2/64 | 235,328 | 65,536 | 46,137,344 | 685,648 | 2.914 | 0.01 |
| 2/64 | 235,328 | 131,072 | 92,274,688 | 601,760 | 2.557 | 0.01 |
| 4/128 | 1,063,808 | 512 | 360,448 | 355,837 | 0.334 | 0.99 |
| 4/128 | 1,063,808 | 1,024 | 720,896 | 710,632 | 0.668 | 0.99 |
| 4/128 | 1,063,808 | 2,048 | 1,441,792 | 1,418,799 | 1.334 | 0.98 |
| 4/128 | 1,063,808 | 4,096 | 2,883,584 | 2,703,849 | 2.542 | 0.94 |
| 4/128 | 1,063,808 | 8,192 | 5,767,168 | 3,244,810 | 3.050 | 0.56 |
| 4/128 | 1,063,808 | 16,384 | 11,534,336 | 3,457,478 | 3.250 | 0.30 |
| 4/128 | 1,063,808 | 32,768 | 23,068,672 | 3,464,226 | 3.256 | 0.15 |
| 4/128 | 1,063,808 | 65,536 | 46,137,344 | 3,169,280 | 2.979 | 0.07 |
| 4/128 | 1,063,808 | 131,072 | 92,274,688 | 2,561,616 | 2.408 | 0.03 |
| 8/256 | 6,859,520 | 512 | 360,448 | 355,838 | 0.052 | 0.99 |
| 8/256 | 6,859,520 | 1,024 | 720,896 | 710,649 | 0.104 | 0.99 |
| 8/256 | 6,859,520 | 2,048 | 1,441,792 | 1,419,224 | 0.207 | 0.98 |
| 8/256 | 6,859,520 | 4,096 | 2,883,584 | 2,834,165 | 0.413 | 0.98 |
| 8/256 | 6,859,520 | 8,192 | 5,767,168 | 5,658,691 | 0.825 | 0.98 |
| 8/256 | 6,859,520 | 16,384 | 11,534,336 | 11,289,946 | 1.646 | 0.98 |
| 8/256 | 6,859,520 | 32,768 | 23,068,672 | 20,493,940 | 2.988 | 0.89 |
| 8/256 | 6,859,520 | 65,536 | 46,137,344 | 19,794,916 | 2.886 | 0.43 |
Text results (16M-param model, ref trained on 262,144 windows)
| N | train NLL (bits/window) | ref NLL | mem_U/window | mem_U total | F1 delta | F1 raw |
|---|---|---|---|---|---|---|
| 500 | 794.9 | 815.3 | 20.3 | 10,171 | 0.667 | 0.838 |
| 2,048 | 2.1 | 411.8 | 409.7 | 839,071 | 0.200 | 1.000 |
| 8,192 | 3.6 | 411.4 | 407.7 | 3,340,194 | 0.500 | 1.000 |
| 32,768 | 141.9 | 411.7 | 269.8 | 8,840,403 | 0.800 | 1.000 |
| 131,072 | 412.4 | 411.3 | 12.2 | 1,604,732 | 0.941 | 0.942 |
| 262,144 | 433.9 | 411.0 | 2.4 | 617,305 | 0.970 | 0.970 |
α lower bounds: {"L2d64": {"n_params": 235328, "alpha_lower_bound_bpp": 3.15845968180582, "peak_memorized_bits": 743274.0, "peak_at_N": 16384}, "L4d128": {"n_params": 1063808, "alpha_lower_bound_bpp": 3.256439131873421, "peak_memorized_bits": 3464226.0, "peak_at_N": 32768}, "L8d256": {"n_params": 6859520, "alpha_lower_bound_bpp": 2.9876638234162156, "peak_memorized_bits": 20493939.75, "peak_at_N": 32768}}
- Downloads last month
- 19