gptel + codex-cli (actually guidance needed on how to make it work more smoothly)

[ag91]

Hey thanks again for the useful package: it rocks!

I have been trying codex-cli and I thought it would be great to use it from gptel.
So with a bit of config through mcp.el I got to:

("codex" . (:command "npx"
                                  :args ("-y" "codex" "mcp")
                                  :env (:OPENAI_KEY ,pass)
                                  ))

and it does work, but I think there is something I don't understand: the gptel buffer just stops and the progress notifications get displayed in the Message buffer.

I found the definition that is doing it in mcp.el and I tried to redefine it below: no matter what it doesn't write content in the gptel buffer...
I think I just need a pointer to get my code to write to the buffer and I can probably figure out how to fix also the tool state notification area (it seems to not update at all after calling the tool).

(defun mcp-notification-dispatcher (connection name method params)
  "Handle notifications from MCP server.
CONNECTION is the JSON-RPC connection object. NAME identifies the
server. METHOD is the notification name. PARAMS contains the
notification data."
  (pcase method
    ('notifications/message
     (cond ((or (plist-member (mcp--capabilities connection) :logging)
                (and (plist-member params :level)
                     (plist-member params :data)))
            (cl-destructuring-bind (&key level data &allow-other-keys) params
              (let ((logger (plist-get params :logger)))
                (message "[mcp][%s][%s]%s: %s"
                         name
                         level
                         (if logger
                             (format "[%s]" logger)
                           "")
                         data))))))
    ('codex/event
     (message "%s Received notification: method=%s, params=%s, fsm=" name method params )
     (let* ((msg (plist-get params :msg))
            (type (plist-get msg :type))
            ;; This assumes there's a gptel request active and this is the
            ;; buffer where it was made.
            (info (and-let* ((fsm (with-current-buffer
                                      "*OpenRouter*" ;; forcing the buffer because I couldn't get hold of the fsm -- sort of worked like this
                                    gptel--fsm-last)))
                    (gptel-fsm-info fsm))))
       (when info
         (with-current-buffer (plist-get info :buffer)
           (pcase type
             ('agent_reasoning
              (when-let ((delta (plist-get msg :text)))
                (gptel--display-reasoning-stream delta info)))
             ('agent_reasoning_delta
              (when-let ((delta (plist-get msg :delta)))
                (gptel--display-reasoning-stream delta info)))
             ('agent_message
              (when-let ((delta (plist-get msg :message)))
                (goto-char (point-max))
                (insert delta);; (gptel-curl--stream-insert-response delta info) ;; no matter what, this doesn't write
                ))
             ('agent_message_delta
              (when-let ((delta (plist-get msg :delta)))
                (goto-char (point-max))
                (insert delta)))
             ('exec_command_begin
              (when-let ((command (plist-get msg :command)))
                (goto-char (point-max))
                (insert (concat "\n\n**Running command:**\n\n=sh\n" command "\n=\n\n"))))
             ('exec_command_output_delta
              (when-let* ((chunk (plist-get msg :chunk))
                          (decoded-chunk (progn (require 'base64) (base64-decode-string chunk))))
                (goto-char (point-max))
                (insert  decoded-chunk)
                )))))))
    (_
     (message "%s Received notification: method=%s, params=%s" name method params))))

[jl0jl]

i am going to try the same setup so hpefully i will have more luck! will post if so

[ag91]

oh I somewhat managed with this, needs some extension though:

(defun my/extract (msg type)
  (cond 
   ((equal type "agent_reasoning")
    (when-let ((delta (plist-get msg :text)))
      ;; (gptel--display-reasoning-stream delta info)
      delta
      ))
   ((equal type "agent_reasoning_delta")
    (when-let ((delta (plist-get msg :delta)))
      ;; (gptel--display-reasoning-stream delta info)
      delta
      ))
   ((equal type "agent_message")
    (when-let ((delta (plist-get msg :message)))
      delta))
   ((equal type "agent_message_delta")
    (when-let ((delta (plist-get msg :delta)))
      delta))
   ((equal type "exec_command_begin")
    (when-let ((command (plist-get msg :command)))
      (concat "\n\n**Running command:**\n\n=sh\n" command "\n=\n\n")))
   ((equal type "exec_command_output_delta")
    (when-let* ((chunk (plist-get msg :chunk))
                (decoded-chunk (progn (requaluire 'base64) (base64-decode-string chunk))))
      (goto-char (point-max))
      (insert  decoded-chunk)
      ))
   ((equal type "task_complete")
    (when-let ((last-message (plist-get msg :last_agent_message)))
      last-message)
    )
   (t (warn "Codex type unhandled: Type=%s\nFor message:%s" type msg))
   )
  )

(defun mcp-notification-dispatcher (connection name method params)
  "Handle notifications from MCP server.
CONNECTION is the JSON-RPC connection object. NAME identifies the
server. METHOD is the notification name. PARAMS contains the
notification data."
  (pcase method
    ('notifications/message
     (cond ((or (plist-member (mcp--capabilities connection) :logging)
                (and (plist-member params :level)
                     (plist-member params :data)))
            (cl-destructuring-bind (&key level data &allow-other-keys) params
              (let ((logger (plist-get params :logger)))
                (message "[mcp][%s][%s]%s: %s"
                         name
                         level
                         (if logger
                             (format "[%s]" logger)
                           "")
                         data))))))
    ('codex/event
     (message "%s Received notification: method=%s, params=%s, fsm=" name method params )
     (let* ((msg (plist-get params :msg))
            (type (plist-get msg :type))
            ;; This assumes there's a gptel request active and this is the
            ;; buffer where it was made.
            (info (and-let* ((fsm (with-current-buffer
                                      "*OpenRouter*" ;; TODO 
                                    gptel--fsm-last)))
                    (gptel-fsm-info fsm)))
            )
       ;; (message "hey-- %s " (list msg type))
       (with-current-buffer (plist-get info :buffer) 
         (cond 
          ((equal type "agent_reasoning")
           (when-let ((delta (plist-get msg :text)))
             (gptel--display-reasoning-stream delta info)
             ;; (insert delta)
             ))
          ((equal type "agent_reasoning_delta")
           (when-let ((delta (plist-get msg :delta)))
             (gptel--display-reasoning-stream delta info)
             ;; (insert delta)
             ))
          ((equal type "agent_message")
           (when-let ((delta (plist-get msg :message)))
             (goto-char (point-max))
             ;; (insert delta)
             (gptel-curl--stream-insert-response delta info)
             ))
          ((equal type "agent_message_delta")
           (when-let ((delta (plist-get msg :delta)))
             (gptel-curl--stream-insert-response delta info)
             ;; (insert delta)
             ))
          ((equal type "exec_command_begin")
           (when-let ((command (plist-get msg :command)))
             (goto-char (point-max))
             (insert (concat "\n\n**Running command:**\n\n=sh\n" command "\n=\n\n"))))
          ((equal type "exec_command_output_delta")
           (when-let* ((chunk (plist-get msg :chunk))
                       (decoded-chunk (progn (require 'base64) (base64-decode-string chunk))))
             (goto-char (point-max))
             (insert  decoded-chunk)
             )
           )
          ((equal type "task_complete")
           (when-let* ((last-message (plist-get msg :last_agent_message)))
             (goto-char (point-max))
             (insert  last-message)
             )
           )
          (t (warn "Codex type unhandled: Type=%s\nFor message:%s" type msg))))))
    (_
     (message "%s Received notification: method=%s, params=%s" name method params))))

[rchar01]

Hi,
I'm curious: why not use Emacs + gptel + mcp.el directly? What's lacking in your current gptel + mcp setup that makes you want to instead integrate codex-cli? What features or behavior do you expect that gptel+mcp doesn't provide for you?

[ag91]

uhm, I didn't have time (nor interest for now) to define all the tools needed and context management myself.
codex has been working quite nicely and I was trying to integrate it as a tool in gptel, so I could use a single interface.
I didn't have time yet to properly investigate this venue though.

[jl0jl]

for me, I wanted codex to work with gptel for a unified interface, where i could delegate hard tasks and for aidermacs-like tasks. I also get free usage of codex with chatgpt + subscription...

I am now trying ai-code-interface.el but will set up gptel with with an mcp.el connection to codex too for non-coding tasks

[ag91]

I fixed it myself, I was just wrong on how I was doing the data extraction