SuperAPI AI Directory
Partner with us中文AI newsFavorites

AI learning

1,062 results · Page 13 of 30

爱云资讯

AI learning

Aiyun News is an information platform for the artificial intelligence technology industry, including industry information on artificial intelligence news, artificial intelligence majors, artificial intelligence technology, artificial intelligence robots, artificial intelligence concept stocks, the Internet of Things, cloud computing, and more. It publishes artificial intelligence industry information every day, reports major events in the artificial intelligence industry, follows hot topics in the AI industry, and serves artificial intelligence enterprises.

Read more

A Visual Guide to Mixture of Experts (MoE)

AI learning

This is a visual guide written by Maarten Grootendorst that uses more than 50 illustrations to explain the Mixture of Experts (MoE) technique in large language models (LLMs) in an accessible way. Starting from the feedforward neural network (FFNN) in the traditional Transformer, the article introduces the basic concepts of sparse models and MoE, and focuses on analyzing MoE's two core components: Experts and the Router. The content covers what experts actually learn (they lean toward syntax rather than specific domains), MoE's architectural design, routing mechanisms (such as KeepTopK, Token Choice, Top-1/Top-k routing), load balancing and auxiliary loss, expert capacity and token overflow, and other key issues. The article also introduces the Switch Transformer, which simplifies MoE training, and explores the application of MoE in vision models (such as V-MoE and Soft-MoE). The guide is suitable for readers who want to understand the principles of MoE and grasp the core ideas without complex mathematical derivations.

Read more