Speculative decoding
An inference technique that proposes output tokens with a faster draft process and checks them with the target model. It can reduce generation time when the setup is suitable.
By Mega AI Hub Editorial · Compute & hosting
5,108,791 AI resources indexedWorking toward the world’s largest AI resource index.
An inference technique that proposes output tokens with a faster draft process and checks them with the target model. It can reduce generation time when the setup is suitable.
By Mega AI Hub Editorial · Compute & hosting