CTRL + C or SIGINT regression after 1.111.2

[HAL703]

Describe the Issue
After updating to 1.112.1 (the version after 1.111.2), CTRL + C hangs for a time rather than causing koboldcpp to exit immediately. Before, this did not happen as SIGINT (CTRL + C) was handled correctly. It is probably related to how koboldcpp handles SIGINT. This regression is still present in koboldcpp 1.113, the latest version. Note that this is primarily regarding CLI usage of koboldcpp, I am unsure if this is present with the GUI, but it probably is.

Reproducing
To reproduce, simply start koboldcpp, for example, koboldcpp --contextsize 24576 --threads 32 --gpulayers 43 --usehipblas -fa --smartcache 3 --model <modelpathhere> . Wait until it is finished setting up. Then try to CTRL + C. It will hang for a seemingly random amount of time, anywhere from 2 seconds to more than a dozen. Expected result is to exit immediately, like in 1.111.2.
For example, please see this image:

Please note that the model used does not affect whether this happens, it always happens.

More Info
This regression affects koboldcpp versions 1.112, 1.112.1, 1.112.2, 1.113. During the hang, cpu usage rises to around 2-3% until it finally segfaults. I am not sure why. Relevant sudo dmesg entry:

[42276.486523] python3[6199]: segfault at 1 ip 00007f8f47efb7b0 sp 00007ffe691a03a0 error 4 in koboldcpp_hipblas.so[40fa7b0,7f8f47a06000+13ec000] likely on CPU 0 (core 0, socket 0) [42276.486531] Code: ee 00 48 8b 7d 28 48 8d 45 38 48 39 c7 74 0d 48 8b 45 38 48 8d 70 01 e8 1e 1d ee 00 4c 8b 65 10 48 8b 5d 08 49 39 dc 74 27 90 <48> 8b 3b 48 8d 43 10 48 39 c7 74 0d 48 8b 43 10 48 8d 70 01 e8 f7

System Information:
Operating System: EndeavourOS (Arch Linux)
KDE Plasma Version: 6.6.5
KDE Frameworks Version: 6.26.0
Qt Version: 6.11.1
Kernel Version: 7.0.8-1-cachyos (64-bit) (this bug also happens on the regular linux kernel)
Graphics Platform: Wayland
Processors: 32 × AMD Ryzen 9 9950X 16-Core Processor
Memory: 32 GiB of RAM (30.9 GiB usable)
Graphics Processor: AMD Radeon RX 7900 XT
Manufacturer: Gigabyte Technology Co., Ltd.
Product Name: X870 AORUS ELITE WIFI7

[LostRuins]

Does it happen in 1.112 (not the .1)

[HAL703]

Yes it does as far as I am aware @LostRuins. Let me add that, I thought just saying the additional versions would imply the base, but it doesn't.

[LostRuins]

okay so the last "known good" version is 1.111.2?

[HAL703]

Yes. @LostRuins

[wbruna]

My guess would be Ctrl+C itself still works, but it triggers a cleanup function which causes the issue.

Does it also happen on Vulkan? What about plain CPU?

[LostRuins]

@wbruna do you get this issue too? I have heard scattered reports but cannot conclusively repro it. Does seem to be a relatively new thing

[HAL703]

This still occurs with vulkan, yes. Still occurs with cpu, yes. Just tested it on 1.113.

[wbruna]

@wbruna do you get this issue too? I have heard scattered reports but cannot conclusively repro it. Does seem to be a relatively new thing

I do, but... not always 😕 It does not happen with my usual launching script (which sets env vars, loads a .kcpps file, etc), but I get it with an even simpler command line:

python koboldcpp.py --usehipblas 0 --model Qwen3-0.6B-UD-Q4_K_XL.gguf

[wbruna]

Got something:

Thread 1 "python3.13" received signal SIGSEGV, Segmentation fault.
0x00007ffff55b82b0 in BPETokenizer::~BPETokenizer() () from ./koboldcpp_default.so
(gdb)

It turns out there is more than one BPETokenizer class in the codebase...

https://github.com/LostRuins/koboldcpp/blob/v1.113.1/otherarch/acestep/bpe.h#L267
https://github.com/LostRuins/koboldcpp/blob/v1.113.1/otherarch/sdcpp/tokenizers/bpe_tokenizer.h#L16

[wbruna]

Indeed, this hack fixes it for me:

diff --git a/otherarch/acestep/bpe.h b/otherarch/acestep/bpe.h
index fe5ebc6d1..6da28c0f8 100644
--- a/otherarch/acestep/bpe.h
+++ b/otherarch/acestep/bpe.h
@@ -263,6 +263,8 @@ static std::vector<std::string> gpt2_pre_tokenize(const std::string &text) {
     return chunks;
 }
 
+namespace kcpp_workaround_multiple_BPETokenizer {
+
 // BPE tokenizer struct
 struct BPETokenizer {
     std::unordered_map<std::string, int> vocab;  // token_str -> id
@@ -273,6 +275,9 @@ struct BPETokenizer {
     std::vector<std::string> id_to_str;          // id -> token_str (reverse vocab)
 };
 
+}
+using kcpp_workaround_multiple_BPETokenizer::BPETokenizer;
+
 // Minimal JSON parser for vocab.json ({"str": int, ...})
 static bool load_vocab_json(const std::string &path, std::unordered_map<std::string, int> &vocab) {
     FILE *f = fopen(path.c_str(), "rb");

[LostRuins]

Wow, interesting find. I'll try that fix

[LostRuins]

I added the hacky patch wbruna suggested, running the CI now.

Builds will be ready in 2 hours https://github.com/LostRuins/koboldcpp/releases/tag/rolling

[HAL703]

Testing now, will let you know if I find anything.

[HAL703]

@LostRuins In regards to the current rolling release as of this comment:

Using koboldcpp-linux-x64-nocuda --contextsize 24576 --threads 32 --gpulayers 39 --usecpu -fa --model <modelhere>, it loads fine (yes I am aware that command isn't the best, but it serves the purpose of testing the cpu regardless). When I CTRL + C however, I get this error:

Traceback (most recent call last):
  File "koboldcpp.py", line 11673, in <module>
    main(launch_args=parser.parse_args(),default_args=parser.parse_args([]))
  File "koboldcpp.py", line 10239, in main
    kcpp_main_process(args,global_memory,using_gui_launcher)
  File "koboldcpp.py", line 11468, in kcpp_main_process
    asyncio.run(RunServerMultiThreaded(args.host, args.port, KcppServerRequestHandler))
  File "koboldcpp.py", line 7120, in RunServerMultiThreaded
    exitcounter = 999
KeyboardInterrupt
[PYI-70387:ERROR] Failed to execute script 'koboldcpp' due to unhandled exception!

When I tried vulkan with --usevulkan, I get this error:

^CTraceback (most recent call last):
  File "koboldcpp.py", line 7117, in RunServerMultiThreaded
    time.sleep(10)
KeyboardInterrupt

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "koboldcpp.py", line 11673, in <module>
    main(launch_args=parser.parse_args(),default_args=parser.parse_args([]))
  File "koboldcpp.py", line 10239, in main
    kcpp_main_process(args,global_memory,using_gui_launcher)
  File "koboldcpp.py", line 11468, in kcpp_main_process
    asyncio.run(RunServerMultiThreaded(args.host, args.port, KcppServerRequestHandler))
  File "koboldcpp.py", line 7120, in RunServerMultiThreaded
    exitcounter = 999
KeyboardInterrupt
[PYI-70387:ERROR] Failed to execute script 'koboldcpp' due to unhandled exception!

Mind you, I was NOT using the gui launcher. So I am not sure why it mentions that in the errors.