EXAMPLE 01
Use a conventional FFN.
PYTHON
from mlbricks import SOUP
model = SOUP(
dim=512,
width=1116,
depth=2,
mixer="esa",
ffn="ffn",
)EXAMPLE 02
Mix ESA and Bolt by layer.
PYTHON
model = SOUP(
dim=512,
width=[1116, 1024],
depth=2,
mixer=["esa", "bolt"],
ffn=["saffn", "saffn"],
)EXAMPLE 03
Validate before a training run.
PYTHON
report = model.validate(torch.randn(2, 128, 512))
print(report)EXAMPLE 04
Use recurrent fallback without a prepared fast plan.
If a compatible custom mixer/FFN implements recurrentprefill/decode_stepbut cannot use SOUP's prepared generation plan, call those recurrent methods directly and skipprepare_generation().