code is incorrect

[YZH0905]

This code is completely incorrect, and the code of the cashe part is also incorrect. The acceleration effect shown in the figure is only because the sentence prompt is relatively short and the max_new_token setting is relatively small. When the prompt becomes longer, the running time explodes.

[romsto]

Thanks for the feedback, but "completely incorrect" isn't very helpful here.
The KV cache issues are already documented in Issue #1 and the README. That's why I recommend to switch use_cache=False for now. I didn't have the time yet, but I plan on fixing it.

If you found specific bugs, please share:

• What exactly is wrong?
• What behavior did you see?
• Any suggested fixes.

The core speculative decoding algorithm works as intended within its current limitations (longer sequences require the kvcache to work as intended). Constructive feedback with actual details would be much more useful than just saying "it's wrong".