gptel-gutter-mode

[nleve]

I use gptel all the time, but one pain point for me has been determining whether the point is currently in LLM-response text, or outside it. I don't really like relying on response prefixes, really I want to see which text is LLM-generated more explicitly.

To address this I wrote a gptel-gutter-mode. Inspired by git-gutter-mode and diff-hl-mode, it draws bitmap lines in the fringe when text has either the response or ignore gptel text property.

Screencast_20251005_190059.webm

Would this be of interest to include (as optional) in gptel? To me it doesn't seem worth publishing as a separate package, but I could do that instead if there was interest.

;; ---------------------------------------------------------------------------
;;  gptel-gutter – fringe/margin indicators for response/reasoning lines
;; ---------------------------------------------------------------------------
(defvar-local gptel-gutter--overlays nil
  "List of fringe overlays in current buffer for cleanup.")

;; Define custom fringe bitmap that fills the entire line height
(define-fringe-bitmap 'gptel-gutter-bar
  (make-vector 28 #b01100000)
  nil nil 'center)

(defun gptel-gutter--make-prefix (type)
  "Create fringe prefix string for TYPE ('response or 'ignore)."
  (propertize " " 'display
              `(left-fringe gptel-gutter-bar
                           ,(pcase type
                              ('response 'outline-1)
                              ('ignore 'outline-2)
                              (_ 'outline-1)))))

(defun gptel-gutter--refresh (beg end)
  "JIT-lock function: mark lines containing gptel response/reasoning.
BEG and END delimit the region to refresh."
  (save-excursion
    (save-restriction
      (widen)  ; Ensure we see the whole buffer
      (goto-char beg)
      (beginning-of-line)
      (setq beg (point))

      ;; Extend end to include the full last line
      (goto-char end)
      (end-of-line)
      (setq end (min (1+ (point)) (point-max)))

      ;; Process region by property changes for efficiency
      (goto-char beg)
      (let ((pos beg))
        (while (< pos end)
          (let* ((next-change (or (next-single-property-change pos 'gptel nil end) end))
                 (val (get-text-property pos 'gptel)))
            (when (memq val '(response ignore))
              ;; Process each line in this property region
              (save-excursion
                (goto-char pos)
                (while (and (< (point) next-change) (< (point) end))
                  (let* ((bol (line-beginning-position))
                         (eol (line-end-position))
                         ;; Overlay must extend past EOL to catch line-prefix
                         (overlay-end (min (1+ eol) (point-max)))
                         ;; Check for existing overlay at BOL
                         (existing-ov (cl-find-if
                                       (lambda (ov) (overlay-get ov 'gptel-gutter))
                                       (overlays-at bol))))
                    (if existing-ov
                        ;; Update existing overlay
                        (progn
                          (move-overlay existing-ov bol overlay-end)
                          ;; Update prefix in case type changed
                          (let ((prefix (gptel-gutter--make-prefix val)))
                            (overlay-put existing-ov 'line-prefix prefix)
                            (overlay-put existing-ov 'wrap-prefix prefix)))
                      ;; Create new overlay
                      (let ((ov (make-overlay bol overlay-end nil t nil))
                            (prefix (gptel-gutter--make-prefix val)))
                        (overlay-put ov 'gptel-gutter t)
                        (overlay-put ov 'gptel-gutter-type val)
                        (overlay-put ov 'line-prefix prefix)
                        (overlay-put ov 'wrap-prefix prefix)
                        (overlay-put ov 'evaporate t)
                        (push ov gptel-gutter--overlays))))
                  (forward-line 1))))
            (setq pos next-change)))))
  
  `(jit-lock-bounds ,beg . ,end)))

(defun gptel-gutter--clear ()
  "Remove all gptel fringe overlays in current buffer."
  (mapc #'delete-overlay gptel-gutter--overlays)
  (setq gptel-gutter--overlays nil)
  ;; Also remove any orphaned overlays
  (remove-overlays (point-min) (point-max) 'gptel-gutter t))

(defun gptel-gutter--cleanup-removed-properties (beg end)
  "Remove fringe overlays where gptel property was removed.
Called after text changes to clean up stale overlays."
  (dolist (ov (overlays-in beg end))
    (when (overlay-get ov 'gptel-gutter)
      (let ((ov-start (overlay-start ov)))
        ;; Check if overlay still covers text with gptel property
        (unless (and ov-start
                     (memq (get-text-property ov-start 'gptel)
                           '(response ignore)))
          (delete-overlay ov)
          (setq gptel-gutter--overlays
                (delq ov gptel-gutter--overlays)))))))

(defun gptel-gutter--force-refresh ()
  "Force refresh of all fringe indicators."
  (when gptel-gutter-mode
    (gptel-gutter--clear)
    (jit-lock-refontify (point-min) (point-max))))

(defun gptel-gutter--stream-update ()
  "Update fringe indicators during streaming response.
Uses jit-lock for efficient incremental updates."
  (when gptel-gutter-mode
    ;; Let jit-lock handle the refresh efficiently
    ;; This will trigger gptel-gutter--refresh for visible regions
    (jit-lock-refontify)))

(defun gptel-gutter--after-change (beg end _len)
  "After-change function to clean up stale overlays.
BEG and END delimit the changed region."
  (when gptel-gutter-mode
    ;; Clean up any overlays in changed region that lost their property
    (gptel-gutter--cleanup-removed-properties beg end)))

(defun gptel-gutter--kill-buffer-cleanup ()
  "Clean up fringe overlays when buffer is killed."
  (when gptel-gutter-mode
    (gptel-gutter--clear)))

(define-minor-mode gptel-gutter-mode
  "Show fringe indicators for AI response/reasoning lines.

This mode adds visual indicators in the left fringe for lines
that contain AI responses or internal reasoning. Lines with the
'response property show in one color, 'ignore in another.

The indicators appear on both the main line and any visual
continuation lines when text wraps."
  :global nil
  (if gptel-gutter-mode
      (progn
        ;; Register with jit-lock for efficient fontification
        (jit-lock-register #'gptel-gutter--refresh)
        
        ;; Initial refresh of entire buffer
        (gptel-gutter--refresh (point-min) (point-max))
        
        ;; Hook into streaming updates
        (add-hook 'gptel-post-stream-hook #'gptel-gutter--stream-update nil t)
        
        ;; Clean up stale overlays after changes
        (add-hook 'after-change-functions #'gptel-gutter--after-change nil t)
        
        ;; Clean up on buffer kill
        (add-hook 'kill-buffer-hook #'gptel-gutter--kill-buffer-cleanup nil t))
    
    ;; Cleanup when mode is disabled
    (jit-lock-unregister #'gptel-gutter--refresh)
    (remove-hook 'gptel-post-stream-hook #'gptel-gutter--stream-update t)
    (remove-hook 'after-change-functions #'gptel-gutter--after-change t)
    (remove-hook 'kill-buffer-hook #'gptel-gutter--kill-buffer-cleanup t)
    (gptel-gutter--clear)))

(add-hook 'gptel-mode-hook #'gptel-gutter-mode)

[karthink]

@nleve I love this! We can definitely add it to gptel as an optional feature.

However, I gave the code only a cursory look and want to go over the implementation more carefully. I'm quite wary of adding gptel code to after-change-functions, font-lock, jit-lock etc because this tends to slow down Emacs more than I'd like. In particular, this code adds to both jit-lock and after-change-functions, which seems odd. Maybe there is a reason this is required -- I'll know better after I have time to study it in detail.

[nleve]

It did turn out the after-change-functions hook was unnecessary so I removed it, thank you for mentioning that. I had wound up with both after trying to address an edge case, but triggering jit-lock-refontify on gptel-post-stream-hook alone seems to work fine.

I removed the gptel-gutter--overlays tracking list as well, since it's probably overkill and could just be added back later if really needed.

Preferably I'd also get the margin indicators working for non-GUI emacs, but haven't tried that yet. Also may want to make the fringe bitmap and fringe colors customizable.

I'm also not certain that the (widen) in gptel-gutter--refresh will play nicely with a narrowed org buffer, haven't tested that yet as I usually just use markdown.

I'll wait to open a PR in case you have other suggestions, but here is the latest:

;; ---------------------------------------------------------------------------
;;  gptel-gutter – fringe/margin indicators for response/reasoning lines
;; ---------------------------------------------------------------------------

;; Define custom fringe bitmap that fills the entire line height
(define-fringe-bitmap 'gptel-gutter-bar
  (make-vector 28 #b01100000)
  nil nil 'center)

(defun gptel-gutter--make-prefix (type)
  "Create fringe prefix string for TYPE ('response or 'ignore)."
  (propertize " " 'display
              `(left-fringe gptel-gutter-bar
                           ,(pcase type
                              ('response 'outline-1)
                              ('ignore 'outline-2)
                              (_ 'outline-1)))))

(defun gptel-gutter--update-line-overlay (bol eol val)
  "Create or update overlay for line from BOL to EOL with prefix for VAL."
  (let* ((overlay-end (min (1+ eol) (point-max)))
         (existing-ov (cl-find-if
                       (lambda (ov) (overlay-get ov 'gptel-gutter))
                       (overlays-at bol)))
         (prefix (gptel-gutter--make-prefix val)))
    (if existing-ov
        (progn
          (move-overlay existing-ov bol overlay-end)
          (overlay-put existing-ov 'line-prefix prefix)
          (overlay-put existing-ov 'wrap-prefix prefix)
          (overlay-put existing-ov 'gptel-gutter-type val))
      (let ((ov (make-overlay bol overlay-end nil t nil)))
        (overlay-put ov 'gptel-gutter t)
        (overlay-put ov 'gptel-gutter-type val)
        (overlay-put ov 'line-prefix prefix)
        (overlay-put ov 'wrap-prefix prefix)
        (overlay-put ov 'evaporate t)))))

(defun gptel-gutter--refresh (beg end)
  "JIT-lock function: mark lines containing gptel response/reasoning.
BEG and END delimit the region to refresh."
  (save-excursion
    (save-restriction
      (widen)  ; Ensure we see the whole buffer
      (goto-char beg)
      (beginning-of-line)
      (setq beg (point))

      ;; Extend end to include the full last line
      (goto-char end)
      (end-of-line)
      (setq end (min (1+ (point)) (point-max)))

      ;; Step 1: Clean up stale overlays in the region
      (dolist (ov (overlays-in beg end))
        (when (overlay-get ov 'gptel-gutter)
          (let* ((ov-start (overlay-start ov))
                 (ov-end (overlay-end ov)))
            (unless (and ov-start
                         (or (text-property-any ov-start ov-end 'gptel 'response)
                             (text-property-any ov-start ov-end 'gptel 'ignore)))
              (delete-overlay ov)))))

      ;; Step 2: Add/update overlays where properties exist
      (goto-char beg)
      (let ((pos beg))
        (while (< pos end)
          (let* ((next-change (or (next-single-property-change pos 'gptel nil end) end))
                 (val (get-text-property pos 'gptel)))
            (when (memq val '(response ignore))
              ;; Process each line in this property region
              (save-excursion
                (goto-char pos)
                (while (and (< (point) next-change) (< (point) end))
                  (let ((bol (line-beginning-position))
                        (eol (line-end-position)))
                    (gptel-gutter--update-line-overlay bol eol val))
                  (forward-line 1))))
            (setq pos next-change))))))
  
  `(jit-lock-bounds ,beg . ,end))

(defun gptel-gutter--clear ()
  "Remove all gptel fringe overlays in current buffer."
  (remove-overlays (point-min) (point-max) 'gptel-gutter t))

(defun gptel-gutter--stream-update ()
  "Update fringe indicators during streaming response.
Uses jit-lock for efficient incremental updates."
  (when gptel-gutter-mode
    ;; Let jit-lock handle the refresh efficiently
    ;; TODO: If possible, replace with (jit-lock-refontify start end) using gptel bounds for better perf
    (jit-lock-refontify)))

(defun gptel-gutter--kill-buffer-cleanup ()
  "Clean up fringe overlays when buffer is killed."
  (when gptel-gutter-mode
    (gptel-gutter--clear)))

(define-minor-mode gptel-gutter-mode
  "Show fringe indicators for gptel response/reasoning lines.

This mode adds visual indicators in the left fringe for lines
that contain gptel responses or internal reasoning. Lines with the
'response property show in one color, 'ignore in another.

The indicators appear on both the main line and any visual
continuation lines when text wraps."
  :global nil
  (if gptel-gutter-mode
      (progn
        ;; Register with jit-lock for efficient fontification
        (jit-lock-register #'gptel-gutter--refresh)
        
        ;; Initial refresh of entire buffer
        (gptel-gutter--refresh (point-min) (point-max))
        
        ;; Hook into streaming updates
        (add-hook 'gptel-post-stream-hook #'gptel-gutter--stream-update nil t)
        
        ;; Clean up on buffer kill
        (add-hook 'kill-buffer-hook #'gptel-gutter--kill-buffer-cleanup nil t))
    
    ;; Cleanup when mode is disabled
    (jit-lock-unregister #'gptel-gutter--refresh)
    (remove-hook 'gptel-post-stream-hook #'gptel-gutter--stream-update t)
    (remove-hook 'kill-buffer-hook #'gptel-gutter--kill-buffer-cleanup t)
    (gptel-gutter--clear)))

(add-hook 'gptel-mode-hook #'gptel-gutter-mode)

[karthink]

This looks better! I actually had the opposite in mind -- if we use after-change-functions instead of jit-lock, we control exactly how updates are handled instead of using jit-lock rules, which can be overreaching. I guess we won't know without testing which approach is better.

I am somewhat certain from past experiments that jit-lock will refontify buffer regions even when scrolling, which is completely unnecessary for us. Using after-change-functions avoids this issue, but then we have to handle the idle-timers etc ourselves.

I'll try modifying your code to use only after-change-functions instead, and we can compare the two versions after.

EDIT: The diff-hl package uses only after-change-functions, that looks like a good reference.

[nleve]

@karthink Thank you, it may very well be that we only need one or the other (or neither), I'm not very familiar with this area. I had run into some weird edge cases during testing (streaming responses & rehydrating a gptel buffer from a file) that seemed to be addressed by doing both. But I imagine it can be simplified. I'll take another crack at it too when I get a chance.

[karthink]

@nleve Over the weekend I played around with this code (the second version without after-change-functions) and produced several slowdowns, including one that made typing difficult.

I fired up the CPU and memory profilers, and from what I can tell the two main issues are using one overlay per line, leading to thousands of overlays (and increased memory usage), and gptel-gutter--update-line-overlay being slow.

I can fix the latter, but I unfortunately I don't think there's a way around using one overlay per line. I wrote a version of this code that uses one overlay across the whole response region, splitting it or joining two of them as needed. This works, but has issues: this overlay can only store one value for line-prefix and wrap-prefix, which causes issues when the response text includes different values for line-prefix and wrap-prefixon different lines.

This can happen if you use visual-wrap-prefix-mode or org-indent-mode, for example:

screencast_20251013T191834.mp4

This may be a fair compromise, all things considered.

---

Also, I can confirm that jit-lock sometimes reruns gptel-gutter--refresh when I scroll. It always reruns it when I open/close an Org block. So I'm favoring using after-change-functions, like diff-hl does, unless gptel-gutter--refresh can detect staleness in the overlay state very cheaply and exit.

[nleve]

Ah, that's good to know about line-prefix and wrap-prefix. I wonder if one overlay per identical-line/wrap-prefix region would be a better approach if it doesn't add too much overhead. I'll play around with that idea a bit.

[karthink]

The single overlay version is very simple (by comparison).

jit-lock version:

(defun gptel-gutter--refresh-2 (beg end)
  "JIT-lock function: mark lines containing gptel response/reasoning.
BEG and END delimit the region to refresh."
  (unless (= beg end)
    (when-let* ((pos (text-property-any beg end 'gptel 'response))
                (pos (1+ pos))
                (from (or (previous-single-property-change pos 'gptel) pos))
                (to (or (next-single-property-change pos 'gptel) end)))
      (if-let* ((ov (cl-loop for o in (overlays-in (max (1- beg) (point-min))
                                                   (min (1+ end) (point-max)))
                             if (overlay-get o 'gptel-gutter)
                             return o)))
          (move-overlay ov from to)
        (setq ov (make-overlay from to nil t))
        (overlay-put ov 'evaporate t)
        (overlay-put ov 'gptel-gutter t)
        ;; (overlay-put ov 'font-lock-face 'gptel-response-face) ;; Can use a face if required
        (let ((prefix (gptel-gutter--make-prefix 'response)))
          (overlay-put ov 'line-prefix prefix)
          (overlay-put ov 'wrap-prefix prefix)))
      `(jit-lock-bounds ,beg . ,end))))

after-change-functions version:

(defun gptel-gutter--after-change (beg end _pre)
  "Mark gptel response region between BEG and END.

Intended to be added to `after-change-functions'."
  (unless (= beg end)
    (when-let* ((pos (text-property-any beg end 'gptel 'response))
                (pos (1+ pos))
                (from (or (previous-single-property-change pos 'gptel) pos))
                (to (or (next-single-property-change pos 'gptel) end)))
      (if-let* ((ov (cl-loop for o in (overlays-in (max (1- beg) (point-min))
                                                   (min (1+ end) (point-max)))
                             if (overlay-get o 'gptel-gutter)
                             return o)))
          (move-overlay ov from to)
        (setq ov (make-overlay from to nil t))
        (overlay-put ov 'evaporate t)
        (overlay-put ov 'gptel-gutter t)
        ;; (overlay-put ov 'font-lock-face 'gptel-response-face)
        (let ((prefix (gptel-gutter--make-prefix 'response)))
          (overlay-put ov 'line-prefix prefix)
          (overlay-put ov 'wrap-prefix prefix))))))

• You only need one of the two, either use (jit-lock-register 'gptel-gutter--refresh-2) or (add-hook 'after-change-functions 'gptel-gutter--after-change 90 t). (The depth of 90 is important for other reasons.)
• I only handled the response value above, but it's easy to add support for ignore and (tool ...).
• It doesn't ever delete the overlays during regular editing, unless they evaporate if a region is deleted. It actually takes a bit of work to get into a situation where deleting an overlay in gptel-gutter--refresh-2 is necessary.
• Similarly, it doesn't merge two overlays when a non-response text region between two response regions is deleted. This is an edge case that needs to be considered.

[karthink]

On the branch feature-gptel-indicator, I've added a gptel-indicator-mode that does fringe highlighting + applies a response face (if configured).

It uses the single overlay per response region method. It's still buggy (some jit-lock issue) but feel free to try it out.

[kaibagley]

I was thinking that using these gutter indicators would be a nice way to indicate text in buffers that is being used as context for a gptel request too. Currently a buffer used as context for gptel will have it's background set to black, but a gutter indicator would be more elegant ( i think). What do you think?

[karthink]

Sure, that sounds like a good idea. There are some disadvantages to the fringe indicator though -- it overwrites out any existing wrap/line-prefix, and it doesn't work in the tty.

[kaibagley]

Yeah, overwrites bookmark/register indicators too I believe.

[nleve]

We could get around it not working in the tty by using the margin instead of fringe I think. I usually have diff-hl-mode enabled in buffers that I'm using as context though, and the two modes would kinda conflict.

[karthink]

I have gptel-indicator-mode working in the tty with margins now:

Will update the branch soon.

[rchar01]

Thanks for the gutter/indicator-mode proposal - this looks like a useful addition.

A few quick points:

1. Will this mode affect users who already use a prefix workflow (e.g., inserting a text prefix like > AI: or ### assistant: for responses)?
2. It's good to see it appears to be purely visual (fringe/margin overlay) and doesn't actually change the content of prompts or responses. That suggests existing workflows should remain compatible.
3. It would help to clarify how this interacts with overlays or text-properties — for example, if someone is parsing responses by looking at visible prefixes or markers (rather than just relying on the content), could the overlay interfere or override those markers?
4. Will this gutter mode mix cleanly with the existing prefix approach? In other words, can I continue using my prefix workflow and enable this indicator mode without conflict (or do I have to choose one or the other)?

Looking forward to trying it out and seeing how it integrates with existing setups!

I'm attaching my example workflow: I have defined faces for the prompt/response prefix and I'm adding a LOGBOOK section below each response for tracking.

[karthink]

1. Will this mode affect users who already use a prefix workflow (e.g., inserting a text prefix like > AI: or ### assistant: for responses)?

No, it should be fine.

2. It's good to see it appears to be purely visual (fringe/margin overlay) and doesn't actually change the content of prompts or responses. That suggests existing workflows should remain compatible.

Yeah.

3. It would help to clarify how this interacts with overlays or text-properties — for example, if someone is parsing responses by looking at visible prefixes or markers (rather than just relying on the content), could the overlay interfere or override those markers?

The indicator overlay follows gptel's understanding of what is a response and what isn't, so it should be perfectly accurate. If, for some reason, your prefixes/visible markers don't agree with that, then the markers are wrong and need to be adjusted.

4. Will this gutter mode mix cleanly with the existing prefix approach? In other words, can I continue using my prefix workflow and enable this indicator mode without conflict (or do I have to choose one or the other)?

Yes, you can use both. They don't interfere/interact in any way.

I'm attaching my example workflow:

Your setup is absolutely beautiful! If you don't mind, could you provide a couple of screenshots I can use in the README? A short screencast (~20-30 sec) would be even better, but screenshots are fine. I'm having a really hard time updating the README with more modern gptel images as the package grows and changes.

I can link the screenshot/video to your gptel configuration, or to a gptel wiki page with your configuration. If you are okay providing them,

• please use an aspect ratio for the media roughly comparable to the current images in the README, close to 4:3.
• please use an example specific to you instead of a generic query.

[rchar01]

Thank you - I really appreciate your quick and clear answers.

I'll clean up my configuration and then share it on both GitHub and Codeberg. I'll also prepare some screenshots (or a short screencast) using a real example from my workflow - just need a little time to polish things before sending them over.

Thanks again!

[karthink]

I have now pushed gptel-highlight-mode to master, the feature has landed. If you were testing this feature you can switch back to master now.

To use it, turn on gptel-highlight-mode. In any buffer, not just dedicated chat buffers.

By default, it uses the margin to highlight responses, not the fringe. This is because the default is expected to work everywhere, including in the tty. To customize this, see gptel-highlight-methods -- you can use the fringe/margin, as well as apply a face to the response region. These faces are customizable.

There are many things not customizable, like the fringe thickness or the margin indicator style. Ignored and tool call regions are highlighted too, currently in non-customizable ways. I'll look into extracting these into user options if people ask for it frequently.

There might still be edge cases. Please test.