Skip to content

MIP-0081 tasks

Ordered delivery of MIP-0081, one PR per row. §n are the MIP's. Issues: marola-dev/marola-ml#26–#30 (rows 1–5), marola-dev/marola-site#94 (row 6), #706 (row 7).

# slug delivers tests (must exist before the PR) depends on
1 eval-seam marola-ml — §5.1: scripts/eval/backend.py, local.py over a pinned lm_eval, just eval <preset> <task> backend.py --self-test (a fake backend round-trips a ModelRef); one real arc_challenge run on tiny, its file committed –
2 results-store marola-ml — §5.2, §5.3: results.schema.json, render.py, check.py, just eval-render, just eval-check check.py --self-test with eval_check_refuses_out_of_stderr and eval_check_accepts_within_stderr; render.py --self-test 1
3 card-from-results marola-ml — §5.5: publish_hf.py --results publish_hf.py --self-test renders a model-index block from row 1's file 2
4 domain-task marola-ml — §5.4: eval/tasks/marola_ocean/ (YAML, utils.py) from the pinned resources' question set the task's scorer matches OceanBenchmark's coverage on three fixed answers 1
5 first-provider marola-ml — §5.7: one provider module, eval-external.yml (dispatch only), the runners job extended to it; the provider's free run checked by just eval-check actionlint; ci.yml's runners job fails on a test branch that adds a pull_request trigger 2, a provider agreement
6 partners-page marola-site — the partners page §5.8 promises its own issue's screenshots 5
7 mip-implemented marola — MIP-0081 flipped to Implemented with every PR mip_graph.py --check 1–6