Hadoop / Spark Experiment Architecture

Correctness boundary

• Stable case IDs are derived from canonical JSON payloads.
• Partition assignment is deterministic and independent of worker order.
• Every result retains solver, seed, repetition and case-set hashes.
• Repeated runs are compared before aggregate claims are published.

Executable Python

from optfin_orlab import BenchmarkCase, HadoopBenchmarkPlanner

planner = HadoopBenchmarkPlanner()
cases = [BenchmarkCase("sum", "tiny", {"values": [1, 2, 3]}, seed=11, repetition=0)]
manifest = planner.write_jsonl_partitions(cases, "parts", partition_count=4)
print(manifest["case_set_sha256"])