fix: OpenAI base_url default and reasoning effort model option.

[avinash2692]

• Fixes OpenAI backend passes invalid reasoning_effort to non-reasoning models #270
• Fixes OpenAI backend is hard to use #274
• Fixes bug where additionalProperties is not set to False on response_formats generated by GenerativeSlot. This caused GenerativeSlot to not work with OpenAI platform's API.

[mergify]

Merge Protections

Your pull request matches the following merge protections and will not be merged until they are valid.

🟢 Enforce conventional commit

Wonderful, this rule succeeded.

Make sure that we follow https://www.conventionalcommits.org/en/v1.0.0/

• title ~= ^(fix|feat|docs|style|refactor|perf|test|build|ci|chore|revert|release)(?:\(.+\))?:

[nrfulton]

LGTM. Requested a comment.

@avinash2692 : after adding the comment please feel free to break glass and merge without a code review.

@avinash2692: FYI, NVM. I'm rolling the fix to #274 into this PR. That fix requires discussion with the rest of the core team to make sure there are no objections or subtle bugs introduced by defaulting to self._base_url = None for the OpenAI backend.

[nrfulton]

Please also add a comment noting that OpenAI doesn't like it when non-reasoning models get reasoning parameters.

[avinash2692]

done!

[nrfulton]

FYI @jakelorocco and @HendrikStrobelt:

We currently default to ollama when the user instantiates the OpenAI backend without a self._base_url. This PR will change that default behavior.

Now, if the user types:

import mellea

mellea.start_session(backend_name="openai", api_key="...")

then the OpenAI SDK's default endpoint will be used.

However, if the user types:

import mellea

mellea.start_session(backend_name="openai")

without an API key argument, then the ollama backend will be used and a warning will be printed.

Note that the user must still explicitly provide an api_key kwarg. Having an OpenAI API key in their ENV is not enough. This should prevent accidental expenses / data leaks while still providing less surprising behavior for users who are using the OpenAI backend.

[nrfulton]

We should also review @jakelorocco 's notes in #141 before merging.

[jakelorocco]

I think we are close to defaults that make sense here. I think if the user specifies a base_url we should always use that base_url (even if no apikey is set). I also wonder if we should default the apikey to ollama in those situations.

Otherwise, we have no way to target arbitrary localhost ports that don't require an apikey.

For example (and this isn't the best since it uses LiteLLM and we have a separate backend for that), LiteLLM has a proxy that you can run locally. This proxy stores the apikey information itself; so you can target an arbitrary localhost port without an apikey.

My proposed solution would be to just set the parameter default values to work for the ollama version (ie api_key="ollama" and base_url="http://localhost:11434/v1"). Then users can override these values. I think this would also allow users to explicitly set api_key / base_url to None and have the underlying OpenAI SDK still automatically pick up their env vars (without the risk of users accidentally incurring expenses).

[nrfulton]

Consensus: just pass the args through to the openai sdk. Don't do argument handling such as this.

[jakelorocco]

We should also review @jakelorocco 's notes in #141 before merging.

This issues fixes everything but the model id selection mentioned in #141. I'm ambivalent on if we actually want to handle that "smartly" or just make users choose the specific name themselves when we pick wrongly. We could develop a heuristic for it based on base_urls, but that seems prone to errors. If others agree, we can close #141 when this PR is merged.

[nrfulton]

We should also review @jakelorocco 's notes in #141 before merging.

This issues fixes everything but the model id selection mentioned in #141. I'm ambivalent on if we actually want to handle that "smartly" or just make users choose the specific name themselves when we pick wrongly. We could develop a heuristic for it based on base_urls, but that seems prone to errors. If others agree, we can close #141 when this PR is merged.

Discussed in leads.

In the initializer, these lines of code need to change:
https://github.com/generative-computing/mellea/blob/e7e161bc14f4ba419d5bcf69954686cbe86ff75b/mellea/backends/openai.py#L144C1-L154C1

New logic:

1. If model_id : str then that self._model_id = model_id is used.
2. If model_id : ModelIdentifier then self._model_id = model_id.openai_name.
3. Delete all references to the self._hf_model_id field and replace with self._model_id.

Also ensure that this is dead code and then delete:

mellea/mellea/backends/openai.py

Lines 1009 to 1029 in e7e161b

	def apply_chat_template(self, chat: list[dict[str, str]]):
	"""Apply the chat template for the model, if such a model is available (e.g., when it can deduce the huggingface model id)."""
	from transformers import AutoTokenizer
	if not hasattr(self, "_tokenizer"):
	assert self._base_url, (
	"The OpenAI Platform does not support adapters. You must specify a _base_url when using adapters."
	)
	match _server_type(self._base_url):
	case _ServerType.LOCALHOST:
	self._tokenizer: "PreTrainedTokenizer" = ( # noqa: UP037
	AutoTokenizer.from_pretrained(self._hf_model_id)
	)
	case _ServerType.OPENAI:
	raise Exception(
	"apply_chat_template is called while targeting a server at openai.com. "
	"This is not supported --- openai.com does not support Activated Lora. "
	"Use a locally served vllm instance. "
	)
	return self._tokenizer.apply_chat_template(chat, tokenize=False)

Resolution assigned to @avinash2692