We trace the router
We record selected experts, routing weights, shared experts, model provenance, layer, and generation step without mixing in results.
MoEscope · an ongoing systems journal
We trace the route.We measure the work.
We are building an inspectable path from a Mixture-of-Experts router’s decision to the hardware workload it creates. Our goal is to make every transformation visible, testable, and reproducible.
Fig. 01 / expert loadTrace ready
256routed experts
6selected / token
2shared experts
01 / The question
What happens after a model decides where a token should go?
Sparse models make a small decision with large consequences: each token is routed to only part of the model, and that choice becomes real work on a GPU.
At the model level, routing looks simple. A router scores experts, selects a few, and assigns each selection a weight. At the systems level, those choices become uneven groups of matrix multiplications, memory movement, padding, scheduling, and measurable latency.
The connection between those two views is usually hidden inside a stack of frameworks, compilers, kernels, and hardware. We want to expose that connection without skipping the difficult middle.
02 / What we are building
One verifiable path from model semantics to hardware evidence.
We describe the work
We turn assignments into expert histograms, grouped matrix shapes, tile counts, padding, and theoretical execution waves.
We test the evidence
We replay the workload against a correct reference and measured GPU backends while keeping losses and assumptions visible.
03 / How we will work
We would rather show an honest partial result than a polished mystery.
Inspectable by default
Every result should lead back to a trace, an environment, and code we can read.
Clear about provenance
We label synthetic, model-captured, compiled, and measured data as different things.
Built in the open
We keep the questions, mistakes, revisions, and reasoning in the record.
04 / Daily journal
A dated record of what we tried, learned, and changed together.
Each entry is a checkpoint rather than a victory lap. We use the journal to preserve what became clearer, what remains uncertain, and what we will test next.