Inference Layers

Model layers optimized for inference.

Components

• Attention - Attention mechanisms
• Activation - Activation functions
• Linear - Linear layers
• LayerNorm - Layer normalization
• RotaryEmbedding - Rotary position embeddings
• Sampler - Token sampling
• EmbedHead - Embedding and head layers