ARCHITECTURE
Configurable layers feed a shared state-and-memory path.
SOUP's layer stack combines a selected sequence mixer with a selected FFN/state path. The architecture then uses its built-in Observer State Memory and SOUP Fusion to connect current hidden representations, evolving state and memory.
HIGH-LEVEL FLOW
| Stage | Role |
|---|---|
| Layer input | Hidden representation enters the configured SOUP layer. |
| Mixer | ESA, Bolt or a compatible configured mixer processes sequence context. |
| FFN / state path | State-aware or conventional feed-forward processing updates the layer representation. |
| Observer State Memory | Maintains the architecture's built-in historical memory signal. |
| SOUP Fusion | Combines hidden, state and memory views for the output representation. |
WIDTHS & BRIDGES
Equal-width models can use the direct path. Non-uniform layer widths are resolved at construction with the required bridges. Validation is exposed separately throughmodel.validate(example)instead of adding repeated checks to the hot forward path.
GENERATION PATH
Normal training andforward()stay unchanged.prepare_generation()creates an inference-oriented plan for compatible built-in components, whileprefill/decode_stepprovide the recurrent public API.