Change pre-trained model?

[ggnicolau]

I'm trying to create a spell checker proof-of-concept (POC) for an e-commerce search engine. We're already using Transformers architecture of other tasks and I thought about trying it also with spell checker.

I've came across this beatiful API and I want to give it a try. I've seen it uses BERT classical pre-trained model. But I need to use a pre-trained model in portuguese (such as 'BERTimbau') or multi-cross lingual (such as miniLM).

It would be good if we could pass the desired pre-trained model as a parameter for the function.

I may be wrong and it's already implemented. Correct me if I'm wrong. Is there an easy solution or where I can choose my pre-trained model without going low-level?

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs.

[R1j1t]

Hey @ggnicolau, apologies for the delayed response. The package supports the passing of custom models to spacy pipeline and/or contextual spellchecker. Can you please refer to the below snippet:

contextualSpellCheck/examples/ja_example.py

Lines 1 to 12 in 88bbbb4

	import spacy
	import contextualSpellCheck
	nlp = spacy.load("ja_core_news_sm")
	nlp.add_pipe(
	"contextual spellchecker",
	config={
	"model_name": "cl-tohoku/bert-base-japanese-whole-word-masking",
	"max_edit_dist": 2,
	},
	)

If this does not solve the issue, please let me know, and we can work from there! Furthermore, from this issue, it seems that documentation (README) can be updated with this use case. I will update the label accordingly!

[hardianlawi]

I think the codes do not work for all pre-trained models. I tried changing bert-base-cased to roberta-large and it did not work for the example in the documentation

[linhuixiao]

Thank you very much. Due to area internet access limitations, the Bert_base_cased model can‘t be loaded from the internet automatically, this code solves my problem that loading the Bert model from the compute disk.

[shoegazerstella]

Hi, I am also trying this model bert-base-multilingual-uncased and it seem not to work.
I am aiming at a multi-language spell-checker.
Any tips on that? thank you!