libdpf/doc/pages/experiment.md

# Logging, statistics, and experiments {#experiment_costs}

Paper and production runs need three things the bare key API does not:
**leveled run logs**, **granular cost statistics**, and **replayable coins**.
All three live in the same measurement stack —
[`dpf/log.hpp`](@ref dpf/log.hpp), [`dpf/run_log.hpp`](@ref dpf/run_log.hpp),
[`dpf::experiment`](@ref dpf/experiment.hpp), and the
[`prg::count`](@ref dpf/prg_count.hpp) / [`thread_work`](@ref dpf/thread_work.hpp)
counters.

Uninstrumented code stays quiet and keeps reading `/dev/urandom`. Logging and
experiments are opt-in. `run_measured`, the battery, and `party_node` call
[`app::start_logging`](@ref dpf/run_log.hpp) and install an experiment covering
the thread that constructs it. `run_parties`, `measure_plan`, and the battery
give party `i` its own stream, `ex.derive_party(i)`, whose master is SHA-256 of
the experiment's master and `i`, and they use it for every trial, warmup
included. A kernel handed to a compute pool draws from its party's stream.
Replaying the master replays every party, as long as each party makes the same
draws in the same order. The party masters are noted as `p0/master`,
`p1/master`, and so on.

## Run log {#run_log}

[`dpf/log.hpp`](@ref dpf/log.hpp) writes one `key=value` line per event to
stderr, syslog, or a file. Nothing is written until `log::configure` runs
(normally through `app::start_logging`), so library code can log freely and a
test that never configures the log stays quiet. Every line starts with
`ts`, `lvl`, `inv` (invocation id), `pid`, `tid`, then `role=<party>` when the
thread has one, then `ev=<event>` and the event's fields. Seed bytes go through
`record::seed` as hex, as a SHA-256 fingerprint, or not at all.

[`app::start_logging`](@ref dpf/run_log.hpp) turns it on and writes the
provenance banner first, at `info`:

| Event | Contents |
| --- | --- |
| `ev=start` | program, cwd, UTC/local start, invocation id shared with `runs.csv` |
| `ev=build` | git rev (`LIBDPF_GIT_REV`), compiler, `NDEBUG`, ISA, sanitizers |
| `ev=host` | hostname, kernel, CPU model, affinity, governor, turbo, memory, load; `ev=if` per interface |
| `ev=argv` | shell-quoted command line |
| `ev=env` | `DPF_*` / `LIBDPF_*` and perf-relevant vars (`LD_PRELOAD`, …) |
| `ev=config` | every `run_config` setting |

After that come seeds with their source, listeners and links (endpoints, peer
authentication, encryption, applied socket options, kernel RTT), each party's
plan and costs, trial statistics, and failures. `ev=end` at exit gives elapsed
time. The settings are `run_config` keys:

```
DPF_LOG_LEVEL=debug  DPF_LOG=file:/tmp/run.log,stderr  DPF_LOG_SEEDS=hash
--log_level=debug    --log=syslog                      --log_seeds=off
```

Levels are `silent`, `error`, `warning`, `info` (the default), `debug`, and
`trace`. With `log_seeds=full` the log holds every master, so anyone who has
the file can regenerate that party's randomness. `hash` prints a SHA-256
fingerprint instead.

## Replayable seeds

```cpp
dpf::experiment ex("keyword_pir");          // fresh 32-byte master
auto master = ex.seed();                    // record for the paper
auto x = dpf::uniform_sample<std::uint64_t>();

auto again = dpf::experiment::replay("keyword_pir", master);
assert(dpf::uniform_sample<std::uint64_t>() == x);
```

While the experiment is installed, every `uniform_fill` / `uniform_sample`
draw on that thread (DPF roots, Beaver default sampler, pads, Shamir, IKNP,
field rejection) comes from AES-CTR of the master. The master itself is always
drawn from OS entropy, even when nested under another experiment.

Constructors that already own a seed note it automatically:

| Type | Seed name |
| --- | --- |
| (master) | `master` |
| `beavers::oracle` | `beavers::oracle` (+ `lane_table`) |
| `buffered_prg` | `buffered_prg` |
| `prg_pad_rng` | `prg_pad_rng` |
| `pseudorandom_root_sampler` | `pseudorandom_root_sampler` |

Call `ex.note_seed("label", value)` for anything else. Replay is
**order-sensitive**: the same party must make the same `uniform_sample` calls.
Indexed `beavers::oracle(seed)` stays the seekable Beaver path; its seed still
appears in the report when constructed under the experiment.

## Measuring a compose plan {#statistics}

```cpp
auto plan = dpf::protocol::fss_point_plan(0);
auto ex = dpf::app::measure_plan("fss_point", plan);
ex.write_csv("/tmp/run1");
```

Or from an application sketch:

```cpp
// Prints rounds, live bytes, wall/CPU, PRG evals, random bytes, seed hex.
// Set DPF_EXPERIMENT_DIR=/tmp/run1 to also emit CSVs.
dpf::app::run_measured("keyword_pir",
    dpf::protocol::keyword_pir_compose_plan(0, depth), 2);
```

What is recorded:

| Metric | Source |
| --- | --- |
| Interactive rounds / DAG depth | `plan::exchange_waves()` / `plan::waves()` |
| Critical path | back-walk of max-`wave_of` inputs |
| Schedule bytes per edge | `slot_bytes` + `wave_channel` |
| Bytes out/in per round | the round's send and receive slots, on its own channel |
| Wall / CPU | `steady_clock` / party 0's thread CPU plus its pool kernels' CPU |
| Symmetric-key blocks | `prg::count(purpose, primitive)` per thread |
| Random bytes | `uniform_fill` TLS counter |
| Seeds | master + every `note_experiment_seed` |

Symmetric-key blocks are counted by purpose and primitive. The purposes are
`expand` (tree expansion, leaf conversion, label and column PRGs), `hash`
(IKNP row hashes, garbled-gate hashes), and `harness` (the experiment's own
seed stream). The primitives are AES-128, AES-256, ChaCha, and LowMC.
`prg_evals` is `expand` plus `hash` over every primitive. Harness blocks are
never part of it. The counters are per thread; a kernel a party hands to a
compute pool (`compute_threads > 0`) is charged to that party when it returns
([`dpf/thread_work.hpp`](@ref dpf/thread_work.hpp)).

## CSV layout

`write_csv(dir)` appends. Every row ends with the id of the process that
wrote it (`invocation`), so rows from different runs into one directory stay
apart. A file whose header differs from this layout is renamed to
`<name>.before-<UTC>.csv` before new rows go in.

- `summary.csv` — one row per run, including `master_seed` hex. `wall_ns`,
  `cpu_ns`, and the counters come from the last (instrumented) trial;
  `median_ns` and `slowest_median_ns` are medians over the timed trials for
  party 0 and for the slowest party.
- `rounds.csv` — per-round deltas and edge name
- `edges.csv` — protocol totals per peer / rss_next / dealer
- `seeds.csv` — every noted seed as hex
- `critical_path.csv` — node id, wave, opcode, effect
- `config.csv` — every `run_config` setting
- `trials.csv` — every party's wall time for each timed trial
- `wire.csv` — party 0's link counters, headers included
- `sym.csv` — symmetric-key blocks by purpose and primitive
- `runs.csv` — the invocation id that ties rows to the run log, and whether
  the master was fresh, provided, or derived

Before any party starts its clock, all parties wait at a start gate until
every one has finished setup.

## The battery

The harness is `party_bench` and `profile_party`. Both spawn `p0`, `p1`, and
`p2` and dial [`net::trio`](@ref dpf/net/trio.hpp), the library's localhost
mesh (`p0-p1`, `p0-p2`, `p1-p2`). A flow's bytes and rounds are the frames on
that mesh. The repeat barrier is not included.

```
party_bench --tag bench --repeat 3 --warmup 1
party_bench --case arith_mul_p11 --repeat 5
profile_party --suite gadget --repeat 3 --warmup 1
```

`--tag bench` is the default list. The gadget rows are tagged `bench` as well,
so they sit on that list. `profile_party --suite gadget` runs only those rows.
`profile_party --suite all` appends them after the core and extreme suites.

A secret index stays a DPF. The gadget rows time what you do after a key, or
on shares the parties already hold:

- `arith_proj_*`, `arith_mul_*`, `arith_thresh_*`, `arith_chain_mul4` —
  Ball–Malkin–Rosulek word garbling. Party 0 sends the evaluator view on the
  p0–p1 link. Party 1 evaluates that view and opens.
- `yao_if_*`, `yao_onehot_*` — stacked and one-hot garbling on the garbler.
  The active branch's tables and the evaluator's labels go across the p0–p1
  channel, and party 1 evaluates those bytes. Base OT stays on the `iknp` tag.
- `flute_d*` — FLUTE. Party 2 deals the mask shares. Parties 0 and 1 exchange
  the online bits and open.
- `shuffle_n*` — three hidden-shuffle passes. Each pass's array is sent on
  the ring and consumed as the next party's inbound. Party 0 opens the sum.

Sizes are the modulus, the branch width, the table width, and the column
length. The manual pages are [a small word](@ref arith_garble_word),
[a public table](@ref flute_lut), [a secret branch](@ref yao_stack), and
[an array of shares](@ref share_shuffle).

See also [Network, parties, and MPC](@ref network_and_mpc),
[Protocol composition](@ref protocol_compose), and the
[application mockups](@ref applications).