October 9, 2023

Mixture of Experts

↯ One big idea to spark your curiosity

If you think about your company’s data lake, it is divided into different departments and sub-groups. This division of data allows for different models to be fine-tuned to meet the needs of each department. Just like how you wouldn’t expect your marketing department to write code, your marketing LLM should be fine tuned to meet a specific set of tasks.

While having smaller models enables better results, having to ask a finance, engineering, product and operations LLM the same question would be tedious and wouldn’t provide a cohesive answer. By employing a Mixture of Experts (MoE) approach, we get the benefits of both large and small models. Similar to how you would ask a group of experts in your company to collaborate on a report, we can ask our models to collaborate on their response.

The Next Million AI Models: A Mixture of Experts, Gradient

Funding

↯ Developer-friendly companies that announced new funding rounds

PandasAI, a Python library that adds generative AI capabilities to pandas, a popular data analysis and manipulation tool, raised $1.1m in pre-Seed funding.

Genie, an open source framework for building production-ready web applications with Julia, raised $1.2m in pre-Seed funding.

KittyCAD, a platform that provides infrastructure for developers building hardware design tools, raised $5m in Seed funding.

Unblocked, an AI tool that gives developers answers to questions specific to their codebase, raised $8.3m in Seed funding.

Gradient, a platform for building on open-source LLMs through a fine-tuning and inference web API, raised $10m in Seed funding.

Rainforest, a payments-as-a-service solution for embedding payments into applications, allowing merchants to accept payments from users, raised $11.75m in Seed funding.

Opsera, a DevOps platform for building, testing, deploying, reporting, and monitoring applications, raised $12m in Series A Extension funding.

Stitch, a payments service provider offering a single API built on direct integrations with multiple banks and networks across South Africa, raised $25m in Series A Extension funding.

Pulumi, an open source infrastructure as code tool for creating, deploying, and managing cloud infrastructure, raised $41m in Series C funding.

Observe, an observability platform that helps developers work across data silos by storing logs, metrics, and traces in a single data lake, raised $50m in Series A3 funding.

Brite, an open banking platform for processing account-to-account (A2A) payments in real time between consumers and online merchants, raised $60m in Series A funding.