以下部分引用 AI 总结,现阶段 AI 仍然有幻觉。请以内容的原文为准。
数据新鲜度群聊 07-14 ✓卖方 07-14 ✓Wrap 断档15天 ✕News 07-15 ✓X 断档12天 ✕页面生成 07-15
PowLU
4 atoms · 跨 1 天 · 首见 2026-05-29 · 最近 2026-05-29
三色: 🟦 fact 4 · 🟥 take 0 · stance ▲4/▼0/◆0
来源: X 4
时态: fresh:4
标签: 好数字:3 · 好思考:1
事实 (fact) (4)
- 2026-05-29 · 设计目标
Replace SwiGLU's explosive quadratic growth with a gentle, controlled growth curve ✅ Compresses extreme numerical dynamic ranges ✅ Eliminates outliers without truncating valid information ✅ Preserves critical nonlinear expressiveness
tags: 好思考 · → daily
- 2026-05-29 · 大模型验证结果
7.9B model trained on 600B tokens; 124B model trained on 800B tokens. Results: Across 17 authoritative evaluation benchmarks, PowLU matches or outperforms SwiGLU/SwiGLU-Clip while achieving rock-solid training stability
tags: 好数字 · → daily
- 2026-05-29 · FP8训练稳定性表现
🔴 PowLU-FP8: Ultra-stable smooth loss (~1.32) with zero spikes
tags: 好数字 · → daily
- 2026-05-29 · FP8训练稳定性对比
🟠 SwiGLU-FP8: Severe uncontrollable loss spikes; 🟢SwiGLU-Clip-FP8: Temporary relief but inevitable spikes; 🔵 SwiGLU-BF16: Stable but sacrifices low-precision training efficiency
tags: 好数字 · → daily
时间轴 (近 20)
- 2026-05-29 ·
fact · 设计目标 · → daily
- 2026-05-29 ·
fact · 大模型验证结果 · → daily
- 2026-05-29 ·
fact · FP8训练稳定性表现 · → daily
- 2026-05-29 ·
fact · FP8训练稳定性对比 · → daily
← 实体目录 · 系统日志