feat(m5stack-tab5): multi-framebuffer DPI panel + LVGL DIRECT rendering into the frame buffers (supersedes #847)

components/m5stack-tab5/Kconfig

@@ -56,4 +56,22 @@ menu "M5Stack Tab5 Configuration"
       PPA-only, and a rotated flush with no PPA available is dropped with an
       error instead of being rotated in software.
 
+  config M5STACK_TAB5_DPI_NUM_FBS
+  int "Number of DPI frame buffers"
+  default 1
+  range 1 3
+  help
+    How many frame buffers the MIPI-DSI DPI panel driver allocates in PSRAM
+    (esp_lcd_dpi_panel_config_t::num_fbs). Each one is 720 x 1280 RGB565 =
+    1.8 MB of PSRAM, so every buffer beyond the first costs 1.8 MB.
+    With 1 (the default) LVGL renders into separate draw buffers and the BSP
+    copies each flushed area into the one frame buffer the panel scans out.
+    With 2 or 3 the application can render straight into the frame buffers
+    and switch whole frames instead of copying them: see
+    M5StackTab5::frame_buffers() and
+    DisplayConfig::render_to_frame_buffers_enabled. LVGL's DIRECT mode uses
+    at most two of them (buffers 1 and 0); a third only serves an
+    application that drives the frame buffers itself. Extra buffers without
+    DIRECT mode buy nothing: LVGL keeps copying into the scanned-out buffer.
+
 endmenu

components/m5stack-tab5/README.md

@@ -101,6 +101,54 @@ auto controller_type = tab5.get_display_controller();
 const char* controller_name = tab5.get_display_controller_name();
 ```
 
+## Rendering into the panel's frame buffers
+
+The DPI panel scans a frame buffer in PSRAM out to the display continuously. By
+default the BSP creates one (`CONFIG_M5STACK_TAB5_DPI_NUM_FBS` = 1): LVGL
+renders into separate draw buffers and every flush copies the area into that
+frame buffer. With 2 (or 3) frame buffers an application can render straight
+into them and show a finished frame by switching the panel to its buffer, with
+no copy:
+
+- `frame_buffers()` returns the panel's frame buffers (after
+  `initialize_lcd()`), each `frame_buffer_size()` bytes, cache-line aligned.
+  `esp_lcd_panel_draw_bitmap()` with a pointer inside one of them makes the
+  panel switch to that buffer instead of copying it.
+- `present_frame(frame_buffer)` switches the panel to a finished frame and
+  waits until the panel scans it out (at most one refresh: about 17 ms, 21 ms
+  on the ILI9881 panel), so the previous buffer is free to draw into:
+  tear-free double buffering. The wait uses the DMA's frame-done event;
+  `on_color_trans_done` cannot serve, because for a frame buffer the driver
+  calls it before the switch. Tear-free therefore depends on that wait
+  succeeding. On a timeout (default 100 ms) the switch stays queued and the
+  call returns `false` rather than stalling: the caller carries on and the
+  next frame may tear. After three consecutive timeouts (a frame-done event
+  that never arrives, e.g. a panel / ESP-IDF combination whose callback does
+  not fire) `present_frame()` disables its wait itself, with a warning, so
+  the UI is not held at one frame per timeout for good; frames may then tear
+  until `set_vsync_enabled(true)` re-arms it. `set_vsync_enabled(false)` skips
+  the wait deliberately. The DIRECT-mode flush ignores the return value for
+  the same reason: it never blocks LVGL on a missing event.
+- `initialize_display({.render_to_frame_buffers_enabled = true})` sets LVGL up in DIRECT
+  mode on two of them, with the BSP's flush calling `present_frame()` at the end
+  of each frame. No draw buffers and no rotation scratch buffer are allocated.
+  The display rotation must stay at 0 (LVGL cannot rotate in DIRECT mode).
+
+```cpp
+// menuconfig: M5Stack Tab5 Configuration -> Number of DPI frame buffers = 2
+auto &tab5 = espp::M5StackTab5::get();
+tab5.initialize_lcd();
+tab5.initialize_display({.render_to_frame_buffers_enabled = true});
+```
+
+Measured on a Tab5 in an LVGL application (full-screen redraw, 720 x 1280
+RGB565): 75.6 ms with full-frame double draw buffers copied into one frame buffer,
+48.6 ms rendering into two frame buffers. Memory: two frame buffers take 3.7 MB of
+PSRAM, against 5.5 MB for one frame buffer plus two full-frame draw buffers (and
+the 1.8 MB rotation scratch buffer is not needed either). The frame buffers are
+cache-line aligned, so they also satisfy a raised `LV_DRAW_BUF_ALIGN` (e.g. 128
+for the PPA draw unit), which a plain `heap_caps_malloc()` draw buffer may not.
+
 ## Example
 
 The [example](./example) shows how to use the `espp::M5StackTab5` hardware abstraction component to initialize and use various subsystems of the Tab5.

components/m5stack-tab5/example/README.md

@@ -71,6 +71,16 @@ those tasks than the defaults assume:
   your own project on this example (or reproduce its functionality), keep
   this override in your `sdkconfig` / `sdkconfig.defaults`.
 
+### Rendering into the panel's frame buffers
+
+Set **M5Stack Tab5 Configuration -> Number of DPI frame buffers** to 2 and
+enable **M5Stack Tab5 Example Configuration -> Render LVGL straight into the
+panel's frame buffers** to run the example with
+`initialize_display({.render_to_frame_buffers_enabled = true})`: LVGL renders in DIRECT
+mode into the panel's frame buffers instead of into two full-frame PSRAM draw
+buffers that are copied on every flush. The rotate button does nothing in this
+mode (LVGL cannot rotate in DIRECT mode). See the component README.
+
 ## Hardware Requirements
 
 - M5Stack Tab5 development board

components/m5stack-tab5/example/main/Kconfig.projbuild

@@ -0,0 +1,19 @@
+menu "M5Stack Tab5 Example Configuration"
+
+    config M5STACK_TAB5_EXAMPLE_RENDER_TO_FRAME_BUFFERS
+        bool "Render LVGL straight into the panel's frame buffers"
+        depends on M5STACK_TAB5_DPI_NUM_FBS > 1
+        default n
+        help
+            Initialize the display with
+            DisplayConfig::render_to_frame_buffers_enabled: LVGL renders in DIRECT
+            mode straight into the DPI panel's frame buffers and a finished
+            frame is shown by switching the panel to it, instead of copying
+            LVGL's draw buffers into the one frame buffer the panel scans
+            out. Saves the two draw buffers and the rotation scratch buffer
+            (PSRAM) and the copy on every flush. Needs
+            CONFIG_M5STACK_TAB5_DPI_NUM_FBS >= 2 (M5Stack Tab5 Configuration).
+            The rotate button does nothing in this mode: LVGL cannot rotate in
+            DIRECT mode.
+
+endmenu

components/m5stack-tab5/example/main/gui.cpp

@@ -417,16 +417,27 @@ void Gui::init_circle_layer() {
 }
 
 bool Gui::update(std::mutex &m, std::condition_variable &cv) {
+  [[maybe_unused]] uint32_t next_ms = 16;
   {
     std::lock_guard<std::recursive_mutex> lock(mutex_);
-    lv_task_handler();
+    next_ms = lv_task_handler();
     // keep the audio volume label in sync with the live BSP state, so the
     // first press of a volume button doesn't appear to jump from a stale
     // default (the values are set by app_main after the Gui is constructed)
     update_audio_label();
   }
   std::unique_lock<std::mutex> lock(m);
+#if CONFIG_M5STACK_TAB5_EXAMPLE_RENDER_TO_FRAME_BUFFERS
+  // The flush already waits for the panel's frame-done event, which paces a
+  // frame that has something to draw; here only sleep until LVGL's next timer
+  // is due (at least 1 ms so an idle LVGL never spins). lv_timer_handler()
+  // returns LV_NO_TIMER_READY when nothing is scheduled, which would sleep
+  // forever: cap that case at the display refresh period.
+  const uint32_t sleep_ms = next_ms == LV_NO_TIMER_READY ? LV_DEF_REFR_PERIOD : next_ms;
+  cv.wait_for(lock, std::chrono::milliseconds(std::max<uint32_t>(sleep_ms, 1)));
+#else
   cv.wait_for(lock, std::chrono::milliseconds(16));
+#endif
   return false; // don't stop the task
 }
 
@@ -541,6 +552,11 @@ bool Gui::draw_page_active() {
 }
 
 void Gui::next_rotation() {
+#if CONFIG_M5STACK_TAB5_EXAMPLE_RENDER_TO_FRAME_BUFFERS
+  // LVGL renders into the panel's frame buffers in DIRECT mode, which cannot
+  // rotate.
+  logger_.warn("Rotation is not available while rendering into the frame buffers");
+#else
   std::lock_guard<std::recursive_mutex> lock(mutex_);
   clear_circles_impl();
   auto rotation = lv_display_get_rotation(lv_display_get_default());
@@ -553,6 +569,7 @@ void Gui::next_rotation() {
   lv_obj_set_size(circle_layer_, width, height);
   lv_obj_align(circle_layer_, LV_ALIGN_CENTER, 0, 0);
   lv_obj_invalidate(circle_layer_);
+#endif
 }
 
 void Gui::cycle_brightness() {

components/m5stack-tab5/example/main/m5stack_tab5_example.cpp

@@ -94,12 +94,21 @@ extern "C" void app_main(void) {
   // repaints at once rather than wiping progressively. Partial updates still
   // only render/flush their dirty area, so the larger buffer costs nothing
   // there (just PSRAM).
-  logger.info("Initializing display...");
+  logger.info("Initializing display ({} DPI frame buffers)...", tab5.frame_buffers().size());
+#if CONFIG_M5STACK_TAB5_EXAMPLE_RENDER_TO_FRAME_BUFFERS
+  // LVGL renders straight into the panel's frame buffers (DIRECT mode): no
+  // draw buffers, no rotation scratch buffer, and no copy per flush.
+  if (!tab5.initialize_display({.render_to_frame_buffers_enabled = true})) {
+    logger.error("Failed to initialize display!");
+    return;
+  }
+#else
   auto pixel_buffer_size = tab5.display_width() * tab5.display_height();
   if (!tab5.initialize_display(pixel_buffer_size)) {
     logger.error("Failed to initialize display!");
     return;
   }
+#endif
 
   // make the filter we'll use for the IMU to compute the orientation
   static constexpr float angle_noise = 0.001f;

components/m5stack-tab5/include/m5stack-tab5.hpp

@@ -1,5 +1,6 @@
 #pragma once
 
+#include <array>
 #include <chrono>
 #include <cmath>
 #include <functional>
@@ -213,11 +214,41 @@ class M5StackTab5 : public BaseComponent {
   /// \return true if the LCD was successfully initialized, false otherwise
   bool initialize_lcd();
 
+  /// How initialize_display() sets LVGL up
+  struct DisplayConfig {
+    /// Size of each of LVGL's two draw buffers, in pixels. They are allocated
+    /// in PSRAM, as is the rotation scratch buffer of the same size. Ignored
+    /// with render_to_frame_buffers_enabled.
+    size_t pixel_buffer_size{1280 * 720 / 10};
+    /// Let LVGL render in DIRECT mode straight into the DPI panel's frame
+    /// buffers (see frame_buffers()) instead of into separate draw buffers.
+    /// No draw buffers and no rotation scratch buffer are allocated, and a
+    /// finished frame is shown by switching the panel to its frame buffer
+    /// instead of copying it. Needs initialize_lcd() first and
+    /// CONFIG_M5STACK_TAB5_DPI_NUM_FBS >= 2.
+    /// The BSP's flush shows each frame with present_frame(), so LVGL never
+    /// draws into the buffer the panel is still scanning out (no tearing).
+    /// \note The display rotation must stay at 0: LVGL cannot rotate in DIRECT
+    ///       mode. Pixels written with write_lcd_lines() are lost at the next
+    ///       frame switch.
+    /// \note If you install your own LVGL flush callback, call
+    ///       lv_display_flush_ready() once for every call of it (LVGL waits
+    ///       for it before flushing the next area), or register a wait
+    ///       callback with lv_display_set_flush_wait_cb(); the transfer-done
+    ///       interrupt no longer does it.
+    bool render_to_frame_buffers_enabled{false};
+  };
+
   /// Initialize the LVGL display
   /// \param pixel_buffer_size The size of the pixel buffer
   /// \return true if the display was successfully initialized, false otherwise
   bool initialize_display(size_t pixel_buffer_size = 1280 * 720 / 10);
 
+  /// Initialize the LVGL display
+  /// \param config How to set LVGL up (draw buffers, or the panel's frame buffers)
+  /// \return true if the display was successfully initialized, false otherwise
+  bool initialize_display(const DisplayConfig &config);
+
   /// Initialize the GT911 multi-touch controller
   /// \param callback The touchpad callback
   /// \return true if the touchpad was successfully initialized, false otherwise
@@ -303,6 +334,90 @@ class M5StackTab5 : public BaseComponent {
   /// \return A shared pointer to the display driver
   const std::shared_ptr<DisplayDriver> &display_driver() const { return display_driver_; }
 
+  /// Most frame buffers the DPI panel driver supports (ESP-IDF's limit)
+  static constexpr size_t max_frame_buffers = 3;
+
+  /// Get the DPI panel's frame buffers
+  /// \return The CONFIG_M5STACK_TAB5_DPI_NUM_FBS frame buffers initialize_lcd()
+  ///         got from the panel driver, each frame_buffer_size() bytes
+  ///         (display_width() x display_height() RGB565 pixels) in PSRAM and
+  ///         cache-line aligned; empty until initialize_lcd() succeeded
+  /// \note The panel scans one of them out all the time. Passing a pointer
+  ///       inside one of them to esp_lcd_panel_draw_bitmap() makes the panel
+  ///       switch to that buffer (after a cache write-back) instead of copying
+  ///       it. This lets an application render (e.g. LVGL in DIRECT mode)
+  ///       straight into the frame buffers; see also
+  ///       DisplayConfig::render_to_frame_buffers_enabled.
+  /// \note If you install your own LVGL flush callback, call
+  ///       lv_display_flush_ready() once for every call of it (LVGL waits
+  ///       for it before flushing the next area), or register a wait
+  ///       callback with lv_display_set_flush_wait_cb(); the transfer-done
+  ///       interrupt no longer does it.
+  std::span<void *const> frame_buffers() const {
+    // frame_buffers_ is written before initialize_lcd() opens the gate
+    if (!lcd_initialized_.load(std::memory_order_acquire)) {
+      return {};
+    }
+    return {frame_buffers_.data(), num_frame_buffers_};
+  }
+
+  /// Get the size of one DPI frame buffer
+  /// \return The size of one frame buffer in bytes
+  static constexpr size_t frame_buffer_size() {
+    return display_width_ * display_height_ * sizeof(Pixel);
+  }
+
+  /// Show a whole frame that is already in one of the panel's frame buffers,
+  /// and wait until the panel scans it out
+  ///
+  /// The tear-free way to switch frame buffers, e.g. at the end of an LVGL
+  /// DIRECT-mode frame. esp_lcd_panel_draw_bitmap() on a frame buffer only
+  /// writes the cache back and records the buffer as the next one to scan
+  /// out; the panel's DMA switches to it when it starts its next frame.
+  /// Until then it is still reading the previous buffer, so drawing into that
+  /// one at once would tear. This waits for the DMA's frame-done event (the
+  /// moment it restarts on the new buffer), after which the previous buffer
+  /// is free. (on_color_trans_done cannot be used for this: for a frame
+  /// buffer the driver calls it synchronously, before the switch.)
+  ///
+  /// \param frame_buffer A pointer inside one of frame_buffers() (its start, or any
+  ///        address in it: the buffer that contains it is shown)
+  /// \param timeout_ms How long to wait for the switch
+  /// \return true once the panel scans the frame out, or at once if the wait
+  ///         is disabled or unavailable (see set_vsync_enabled()); false if
+  ///         the panel has only one frame buffer (CONFIG_M5STACK_TAB5_DPI_NUM_FBS
+  ///         must be >= 2: with one there is nothing to switch, and no frame
+  ///         that is safe to draw into afterwards), \p frame_buffer is not
+  ///         inside one of frame_buffers(), a previous transfer is wedged, or
+  ///         the wait timed out (the switch is still queued; carry on rather
+  ///         than stall). Callers are serialized: one presenter at a time.
+  /// \note Waits at most one panel refresh period (about 17 ms; 21 ms on the
+  ///       ILI9881 panel) when the panel is running - two on ESP-IDF before
+  ///       v5.5.5 / v6.0.3, whose frame-done event is the DSI bridge's VSYNC
+  ///       rather than the DMA's re-arm (see present_frame() in video.cpp).
+  ///       Do not call it from an ISR.
+  bool present_frame(const void *frame_buffer, uint32_t timeout_ms = 100);
+
+  /// Enable or disable the wait in present_frame()
+  /// \param enabled false: present_frame() queues the switch and returns at
+  ///        once (faster, but the next frame can tear). present_frame()
+  ///        disables the wait itself after a few consecutive timeouts (a
+  ///        frame-done event that never arrives); pass true to arm it again.
+  ///        Re-arming starts a new outage episode: the consecutive-timeout
+  ///        count and the once-per-episode timeout warning are reset, so a
+  ///        later stall is logged again.
+  void set_vsync_enabled(bool enabled) {
+    vsync_timeouts_in_a_row_ = 0;
+    if (enabled) {
+      vsync_timeout_logged_.store(false, std::memory_order_relaxed);
+    }
+    vsync_enabled_.store(enabled, std::memory_order_relaxed);
+  }
+
+  /// Whether present_frame() waits for the frame-buffer switch
+  /// \return true if the wait is enabled (the default)
+  bool vsync_enabled() const { return vsync_enabled_.load(std::memory_order_relaxed); }
+
   /// Write lines to the LCD
   /// \param xs The x start coordinate
   /// \param ys The y start coordinate
@@ -1106,9 +1221,19 @@ class M5StackTab5 : public BaseComponent {
   // buffer + draw_bitmap copy (which doubles the PSRAM traffic and can starve
   // the DSI scan-out DMA into FIFO underruns / on-screen streaking). Null when
   // unavailable, in which case flush() falls back to the scratch-buffer path.
-  void *dpi_framebuffer_{nullptr};
+  // Null when the PPA direct-to-framebuffer path is unavailable. Moves with
+  // present_frame() to the buffer the panel scans out (written under
+  // panel_op_mutex_, read on the LVGL task by flush()).
+  std::atomic<void *> dpi_framebuffer_{nullptr};
   size_t dpi_framebuffer_bytes_{0};
 
+  // The DPI panel's frame buffers (frame_buffers()), filled by initialize_lcd()
+  std::array<void *, max_frame_buffers> frame_buffers_{};
+  size_t num_frame_buffers_{0};
+  // initialize_display() set LVGL up in DIRECT mode on the frame buffers
+  // (DisplayConfig::render_to_frame_buffers_enabled)
+  bool direct_render_enabled_{false};
+
   // Display controller detection
   DisplayController display_controller_{DisplayController::UNKNOWN};
 
@@ -1143,6 +1268,26 @@ class M5StackTab5 : public BaseComponent {
   // Usually ISR context; the DPI driver's CPU copy path calls it synchronously
   // on the issuing task, so it picks the semaphore API by xPortInIsrContext().
   void complete_transfer(BaseType_t *higher_priority_task_woken);
+  // Frame-done ISR (on_frame_buf_complete, on_refresh_done before ESP-IDF
+  // v5.5.5 / v6.0.3): the DMA has restarted on the frame buffer queued last.
+  // Gives frame_done_sem_ for present_frame().
+  static bool notify_frame_done(esp_lcd_panel_handle_t panel, esp_lcd_dpi_panel_event_data_t *edata,
+                                void *user_ctx);
+  // Given once per panel refresh by notify_frame_done() (only with more than
+  // one frame buffer); present_frame() waits on it
+  SemaphoreHandle_t frame_done_sem_{nullptr};
+  // Serializes present_frame() end to end (queue the switch + wait for the
+  // frame-done event): the event has one token and one waiter per switch.
+  std::mutex present_mutex_;
+  std::atomic<bool> vsync_enabled_{true};
+  // A stalled panel logs one present_frame() timeout, not one per frame
+  std::atomic<bool> vsync_timeout_logged_{false};
+  // Consecutive present_frame() timeouts (only present_frame() touches it,
+  // under present_mutex_); at kVsyncTimeoutsBeforeDisable the wait is
+  // switched off so a frame-done event that never comes cannot hold the UI
+  // at one frame per timeout for good.
+  static constexpr int kVsyncTimeoutsBeforeDisable = 3;
+  int vsync_timeouts_in_a_row_{0};
 
   // DSI write helpers
   void dsi_write_command(uint8_t cmd, std::span<const uint8_t> params, uint32_t flags);