Our flexible and easy-to-develop dllm inference engine is about to release, please pay attention to https://github.com/zhijie-group/Diffulex