Overview

View as Markdown

Neutron for Mojo is an ML inference library targeting Mojo 1.0. It provides tensor operations, quantization formats, neural-network layers, and an inference serving pipeline — all with SIMD-accelerated kernels.

Status: preview. The implementation is complete for pre-1.0 Mojo syntax and is awaiting the Mojo 1.0 compiler release for testing and migration. Treat this as a preview, not a production dependency.

  • Tensors & kernels — typed tensors and SIMD-accelerated operations.
  • Quantization — the supported quantization formats.
  • Inference — model layers, the generation pipeline, graph fusion, and serving.

Project Structure

mojo/
├── tensor/         # Multi-dtype tensor ops, SIMD kernels
├── quant/          # Quantization (NF4, Q4_K, Q8_0, FP8)
├── nn/             # Neural network layers, models, pipelines
├── serve/          # HTTP + text protocol inference server
├── train/          # Training loops, optimizers
├── optim/          # Adam, SGD, AdamW, schedules
├── autograd/       # Automatic differentiation
├── fusion/         # E-graph algebraic optimization
├── data/           # Tokenizers, datasets, data loaders
├── io/             # GGUF, SafeTensors, checkpoints
├── model/          # LLaMA, Phi, Mistral, GPT
├── python/         # Python interop bindings
├── dlpack/         # DLPack tensor exchange
└── cli/            # Inference + benchmark CLI