MIP-0081 tasks¶
Ordered delivery of MIP-0081, one PR per row. §n are the MIP's. Issues: marola-dev/marola-ml#26–#30 (rows 1–5), marola-dev/marola-site#94 (row 6), #706 (row 7).
| # | slug | delivers | tests (must exist before the PR) | depends on |
|---|---|---|---|---|
| 1 | eval-seam | marola-ml — §5.1: scripts/eval/backend.py, local.py over a pinned lm_eval, just eval <preset> <task> |
backend.py --self-test (a fake backend round-trips a ModelRef); one real arc_challenge run on tiny, its file committed |
– |
| 2 | results-store | marola-ml — §5.2, §5.3: results.schema.json, render.py, check.py, just eval-render, just eval-check |
check.py --self-test with eval_check_refuses_out_of_stderr and eval_check_accepts_within_stderr; render.py --self-test |
1 |
| 3 | card-from-results | marola-ml — §5.5: publish_hf.py --results |
publish_hf.py --self-test renders a model-index block from row 1's file |
2 |
| 4 | domain-task | marola-ml — §5.4: eval/tasks/marola_ocean/ (YAML, utils.py) from the pinned resources' question set |
the task's scorer matches OceanBenchmark's coverage on three fixed answers |
1 |
| 5 | first-provider | marola-ml — §5.7: one provider module, eval-external.yml (dispatch only), the runners job extended to it; the provider's free run checked by just eval-check |
actionlint; ci.yml's runners job fails on a test branch that adds a pull_request trigger |
2, a provider agreement |
| 6 | partners-page | marola-site — the partners page §5.8 promises | its own issue's screenshots | 5 |
| 7 | mip-implemented | marola — MIP-0081 flipped to Implemented with every PR | mip_graph.py --check |
1–6 |