CacheBlendPlus

Reimplements CacheBlend's selective KV cache reuse for RAG inference in PyTorch to cut redundant recomputation across document chunks, adds an adaptive recomputation budget driven by divergence scoring, and uses embedding based semantic deduplication with a custom CUDA kernel to profile and speed up the inference pipeline.

Technologies

CUDA · Machine Learning · Python