Inference Basic
导言
RL 涉及到 推理，推理的流程细节不是很明晰。
- warmup，计算kvcache
- chunked prefill，降低prefill的显存
Like this article? Support the author with
Categories
- 1-AI Model Architecture3
- 1-AI Systems18
- 1-Agent Workflow7
- 1-Distributed Parallelism7
- 1-GeneralSkills2
- 1-Operator Development7
- AI6
- Algorithms12
- Architecture41
- Artificial Intelligence74
- Camp1
- Databases2
- HPC1
- Math2
- Network8
- OOW35
- Operating system8
- Overview16
- Programming44
- ProjectRecord3
- Software1
- Thinking13
- Tips9
- Treasure1
- Tutorials119
- Values2
- architecture2
- diary4
- english4
- hardware2
- love1
- math3
- network19
- operating system5
- programming1
- security3
- software23
- thinking9
- tips5
- toLearn56
- values2
Subscribe for updates
follow.it
Archives
- March 20271
- August 20266
- July 202638
- June 20269
- May 202613
- April 20267
- March 20262
- February 20267
- January 20267
- December 202518
- November 20256
- October 20251
- September 20252
- May 20255
- April 20251
- March 20254
- February 20255
- January 20254
- December 20245
- November 20241
- October 20245
- September 202411
- August 20243
- July 20244
- June 20241
- May 20242
- April 20248
- March 20243
- February 20241
- January 202414
- December 202321
- November 202333
- October 202358
- September 202337
- August 202323
- July 202324
- June 202313
- May 202319
- April 20239
- March 20235
- February 20232
- January 20236
- December 20222
- November 20224
- October 20225
- September 20224
- August 20228
- July 20227
- June 20227
- May 202210
- April 20225
- March 20227
- February 20222
- January 20225
- December 20214
- November 20212
- October 202114
- September 202111
- August 202113
- July 202128