Get 20% off this month when you try our services!
EmMiT AI (Efficient Multimodal Multilingual Intelligence Transformer)
- Tagline: "India's Sovereign AI Brain - Built in India
EmMiT AI (Efficient Multimodal Multilingual Intelligence Transformer)
- Tagline: "India's Sovereign AI Brain - Built in India
EmMiT as a frontier-grade,
India-native AI foundation model that rivals GPT-4, Claude 3.5, and DeepSeek-V3.
EmMiT AI is a next‑generation multimodal, multilingual large language model family built in India, for India. Optimized for 22+ Indian languages, STEM, coding, and academic workloads, EmMiT combines Mixture‑of‑Experts efficiency with open‑weight, sovereign deployment on Indian infrastructure
EmMiT‑1B (1.3B dense)Goal: Small, fast model for laptops, single‑GPU servers, and fine‑tuning.ArchitectureLayers: 24Hidden size: 2,048Attention heads: 16 (head dim 128)KV heads: 16 (no GQA compression on smallest model)FFN: SwiGLU with width ≈ 8,192No MoE (pure dense)Compute profileTraining: feasible on 1× high‑end GPU (A100 80GB / 4090) with gradient accumulation.Inference: runs comfortably on 8–12GB GPUs, or CPU with quantization.Use casesOn‑device assistants, edge deployment.Rapid experimentation with EmMiT architecture.Student / research fine‑tuning in Indian languages.
EmMiT‑7B (7B dense)Goal: Strong general model for single‑node GPU servers.ArchitectureLayers: 32Hidden size: 4,096Heads: 32 (head dim 128)KV heads: 8 (4:1 GQA compression)FFN width ≈ 16,384, SwiGLU, dense.Compute profileTraining: 4–8× 24GB GPUs or 1× 80GB with ZeRO/FSDP and smaller batch.Inference: 16GB GPU (FP16), 8GB with 4‑bit quantization.Use casesGeneral EmMiT‑API backend.Fine‑tuning for specific verticals (ed‑tech, support bots).Good trade‑off between quality and cost for SMEs/startups.
EmMiT‑13B (13B dense)Goal: Strong baseline dense model and teacher for distillation.ArchitectureLayers: 40Hidden size: 5,120Heads: 40 (head dim 128)KV heads: 5 (8:1 GQA compression)FFN width ≈ 20,480, SwiGLU.Compute profileTraining: 8× 24GB or 2× 80GB class GPUs.Inference: 24–32GB GPU (FP16), 16GB with 4‑bit.Use casesHigh‑quality multilingual assistant.Baseline for comparing MoE variants.Teacher model for 1B/7B distillation.
Goal: “First serious” MoE model that beats dense‑13B quality at similar or slightly higher cost.ArchitectureSame backbone as 13B dense:40 layers, d_model = 5,120, 40 heads, 5 KV heads.FFN replaced by MoE:Experts per layer: 8Top‑k routing: 2 experts/tokenTotal expert params ≈ 62B, but only ≈ 13B are active per token.Router:Softmax/top‑2 gatingCapacity factor ≈ 1.25Load balancing loss and expert dropout options.Compute profileTraining: 8–16× 80GB GPUs with ZeRO‑3 or FSDP.Inference: active compute roughly like a 13–15B dense model; memory higher due to all experts stored.Use casesMain “workhorse” EmMiT model for hosted API.Fine‑tuning for Indian language + STEM agents.Demonstrator of MoE efficiency vs dense 13B..