DBRX (Databricks, MosaicML)Overview: DBRX, developed by Databricks and MosaicML, is a Mixture-of-Experts model with 36B active parameters (132B total), released in 2024.
Key Features:Uses 16 experts, selecting 4 per inference, offering 65x more expert combinations than similar models.
Excels in retrieval-augmented generation and code-related tasks.
License: Custom open-source license for commercial use.
Use Cases: Enterprise AI, code generation, and data-intensive applications.
Relevance to Superintelligence: DBRX’s efficient MoE architecture is a step toward scalable, compute-efficient systems critical for superintelligence.