update alpaca 13B to 4.1 model

[MichaelCharles]

This resolves issue #350 by updating the version of 13B that is installed.

[Grarie]

You might want to change the name of the resulting file, too. For now this is rather "hacky".
In the same snippet, change the resolve(dir, "ggml-model-q4_0.bin") part.

The problem is, that the script often expects the filename to be the same (the webapp finds only 4_0 named files). I'm not sure how to solve that properly without the need for a rewrite.

[MichaelCharles]

I've made some changes to try to make the references to the model file names in general more generic, but I haven't had a chance to test it yet.

[MichaelCharles]

So, I've tested the updated code and it successfully downloaded the right file, and it seems to be pointing to the right file, however when I run the web app with debug this is the output I'm seeing

/Users/michaelaubrey/dalai/alpaca/main --seed -1 --threads 8 --n_predict 200 --model /Users/michaelaubrey/dalai/alpaca/models/13B/ggml-model-q4_1.bin --top_k 40 --top_p 0.1 --temp 0.1 --repeat_last_n 64 --repeat_penalty 1.3 -p "Once upon a time there lived a girl named Darla, and she
"
exit

The default interactive shell is now zsh.
To update your account to use zsh, please run `chsh -s /bin/zsh`.
For more details, please visit https://support.apple.com/kb/HT208050.
bash-3.2$ /Users/michaelaubrey/dalai/alpaca/main --seed -1 --threads 8 --n_predict 200 --model /Users/michaelaubrey/dalai/alpaca/models/13B/ggml-model-q4_1.bin --top_k 40 --top_p 0.1 --temp 0.1 --repeat_last_n 64 --repeat_penalty 1.3 -p "Once upon a time there lived a girl named Darla, and she
> "
main: seed = 1680573720
llama_model_load: loading model from '/Users/michaelaubrey/dalai/alpaca/models/13B/ggml-model-q4_1.bin' - please wait ...
llama_model_load: invalid model file '/Users/michaelaubrey/dalai/alpaca/models/13B/ggml-model-q4_1.bin' (bad magic)
main: failed to load model from '/Users/michaelaubrey/dalai/alpaca/models/13B/ggml-model-q4_1.bin'
bash-3.2$ exit
exit

[MichaelCharles]

I checked the sha on the file to make sure that it properly downloaded the whole thing, and it seems to check out. I got the sha from https://huggingface.co/Pi3141/alpaca-native-13B-ggml/blob/main/ggml-model-q4_1.bin

shasum -a 256 ggml-model-q4_1.bin | grep -q "7518013326eeb22c409c201f448c0c622d11476f56c2cd5724842930104635b2" && echo "The file has a matching SHA-256 hash value." || echo "The file does not have a matching SHA-256 hash value."
The file has a matching SHA-256 hash value.

[karfly]

@mcaubrey did you solve the problem? I get the same error with your code

[MichaelCharles]

@mcaubrey did you solve the problem? I get the same error with your code

No, as mentioned above, I did solve the problem of it not downloading the model.

However, for whatever reason, the model hosted on Hugging Face does not seem to be in the correct format to use with Dalai.

If you use my updated code, you can download that model, but in the end it still won't execute. You'll likely get the same "bad magic" error that I did.

[devmrh]

any update? i have 13B/ggml-model-q4_1.bin' (bad magic) error too :(