Add support for custom auth plugins

[ducaale]

This PR extends --auth-type to accept plugin-NAME e.g plugin-hmac-sha256 which instructs xh to spawn an executable named xh-plugin-NAME.

Initially, the plugin the receives {"type": "configure"} and it can either respond with {"requires_body": true} or {}.

After that, the plugin is re-spawned for each request and receives the following via stdin:

{
  "type": "before_request",
  "request": {
    "method": "POST",
    "url": "http://example.org/api/v1/hello",
    "headers": [
      { "name": "content-type", "value": "application/json" }
    ],
    "body_base64": "eyJoZWxsbyI6IndvcmxkIn0="
  },
  "auth": [],
  "state": null,
}

The plugin will then modify request by outputting JSON object via stdout:

{
  "remove_headers": ["x-signature"],
  "add_headers": [
    { "name": "x-signature", "value": "12345678" }
  ],
  "set_state": null
}

Some notes:

• request body is buffered similar to digest authentication
• multiple values can be passed to plugin by repeating --auth option e.g --auth-type=plugin-oauth2 --auth=client_id:pluto --auth=client_secret:12345 --auth=scope:42
• env variable XH_AUTH_PLUGIN=1 is passed to plugin
• plugin can be called multiple times if there is a redirect. state can be used to hold arbitrary data.
• auth plugin details are not persisted in session. We also don't save any headers generated by an auth plugin.
• plugin- prefix is not required if full path is specified e.g --auth-type=/usr/bin/oauth2

Resolves #424

[ctron]

I haven't tried it yet, I up slapped together an example of how this could work. Many thanks.

[blyxxyz]

We might want to support plugins for other features besides authentication. Perhaps we can pass a type JSON key to the plugin and recommend checking it? In any case we need to work carefully.

plugin:NAME e.g plugin:hmac-sha256 which instructs xh to spawn an executable named xh-plugin-NAME

This makes sense but it seems useful to be able to specify a plugin by its file path, especially if it's not in $PATH. Maybe we could not apply the xh-plugin- prefix if there's a directory separator in the name? Still feels slightly weird. I wonder what other software does.

stdin

This is probably the best solution but it makes it slightly awkward for plugins to take interactive user input. (On Unix you can still do this by opening /dev/tty.)

"body_base64": "eyJoZWxsbyI6IndvcmxkIn0="

This can be expensive e.g. for large file uploads. How often will plugins need to know the body? Maybe we could figure out a way for the plugin to only request the body if it needs it?

If we leave this out now we can always add it later.

(I see the HMAC example uses it...)

"current_dir": "/home/user"

Can't the plugin look at its own working directory?

[ducaale]

We might want to support plugins for other features besides authentication.

Would an env variable be enough? we're currently passing XH_AUTH_PLUGIN=1 to the plugin

Maybe we could not apply the xh-plugin- prefix if there's a directory separator in the name? Still feels slightly weird. I wonder what other software does.

Sounds good 👍

This is probably the best solution but it makes it slightly awkward for plugins to take interactive user input. (On Unix you can still do this by opening /dev/tty.)

Hmm, I haven't thought about this. Let me check the behaviour in windows as well.

Can't the plugin look at its own working directory?

Yeah, you're right. Let me remove it.

This can be expensive e.g. for large file uploads. How often will plugins need to know the body? Maybe we could figure out a way for the plugin to only request the body if it needs it?

I agree. One thing I was experimented with is to have the plugin configure itself, so if it receives the following:

{
  "next_request": {
    "method": "POST",
    "url": "http://example.org/api/v1/hello",
    "headers": [],
    "body_base64": null
  },
  "auth": [],
  "state": null,
  "current_dir": "/home/user"
  "config": null
}

it can output something like:

{ "set_config": { "requires_body": true } }

Which leads to the plugin being spawned again with the following:

{
  "next_request": {
    "method": "POST",
    "url": "http://example.org/api/v1/hello",
    "headers": [],
    "body_base64": "eyJoZWxsbyI6IndvcmxkIn0="
  },
  "auth": [],
  "state": null,
  "current_dir": "/home/user"
  "config": { "requires_body": true }
}

[blyxxyz]

Would an env variable be enough?

Making it part of the payload seems most natural, you can directly express that as a serde enum or TypeScript tagged union.

Configuring could even be a separate mandatory step, so that we send a payload like {"type": "configure"} (no other fields) and the plugin sends back {"requires_body": true} or just {} if the defaults are fine. Though that increases the amount of boilerplate plugins need and it doubles the startup time.

We could also do JSONL, keep the plugin process running and keep sending it new payloads. (And take care not to deadlock if a naïve plugin e.g. stops listening too early because it doesn't take redirects into account?)

It smells like an ad hoc IPC system, we may be reinventing the wheel...

[ctron]

Would an env variable be enough?

Making it part of the payload seems most natural, you can directly express that as a serde enum or TypeScript tagged union.

The idea of the env-var was to have a "normal" application just define an alias (symlink) and then trigger the plugin behavior by detecting the env-var. That would not work if you would first need to parse stdin.

We could also do JSONL, keep the plugin process running and keep sending it new payloads. (And take care not to deadlock if a naïve plugin e.g. stops listening too early because it doesn't take redirects into account?)

It smells like an ad hoc IPC system, we may be reinventing the wheel...

I think using stdio for things like this and a one shot approach is a fairly common pattern in Unix-ish OSes. Also for Windows. Creating a service and keeping things running makes things rather complicated.

[blyxxyz]

The idea of the env-var was to have a "normal" application just define an alias (symlink) and then trigger the plugin behavior by detecting the env-var. That would not work if you would first need to parse stdin.

I forgot that the env var name contained AUTH. I was imagining a system where an env var is used to tell the program that it's running as a plugin at all and the payload communicates what kind of plugin or which step in the process it's in. (Not sure that's actually better.)

It's unfortunate that env vars get inherited by the plugin's child processes but that's probably unlikely to cause problems in practice. (We could pass it as an argument instead.)

Instead of XH_AUTH_PLUGIN=1 we could do XH_PLUGIN=auth. How useful that is depends on whether the same program might want to implement multiple plugin types.

{ "set_config": { "requires_body": true } }

If we do this, do we also allow other commands in that same response (with the possibility that the plugin modifies headers twice)?

A poorly written plugin might confuse an empty body for a missing body, so maybe we should error if the plugin calls set_config twice to avoid infinite loops? (Could be too paranoid.)

[ctron]

Instead of XH_AUTH_PLUGIN=1 we could do XH_PLUGIN=auth.

Sounds like a good idea!

[ducaale]

Maybe we could not apply the xh-plugin- prefix if there's a directory separator in the name? Still feels slightly weird. I wonder what other software does.

Sounds good 👍

Note that this would be in line with how git finds external helpers. From gitcredentials(7):

1. If the helper string begins with "!", it is considered a shell
   snippet, and everything after the "!" becomes the command.
2. Otherwise, if the helper string begins with an absolute path,
   the verbatim helper string becomes the command.
3. Otherwise, the string "git credential-" is prepended to the
   helper string, and the result becomes the command.

[ducaale]

Does it make sense to still use the plugin: prefix when specifying a full path? e.g --auth-type=plugin:/path/to/plugin vs --auth-type=/path/to/plugin

Update: --auth-type now accepts either plugin-NAME (interpreted as xh-plugin-NAME) or /path/to/plugin

[ducaale]

Couple of questions to think about:

• How can we support multi-step authentication like digest or kerberos? git-credential-helpers use continue keyword to retry a request. According to this talk, it is limited to 3 round trips.
• Should we allow plugins to persist headers if --session=name_or_path is used?
• How should we handle versioning? maybe we can pass xh version as env variable?