Propagating signals that wrap Result to leptos_use::use_event_listener

[TLATER]

Hi, I'm having a bit of a hard time wrapping my head around how signals and resources are supposed to interact.

I'm trying to build a simple media player based on web_sys::HtmlAudioElement and web_sys::AudioContext. Setting up the context takes a bit of boilerplate, so I factored that out into a separate struct; since all of this can only run on the client it all goes into a LocalResource. So far so good:

let player: LocalResource<Result<MediaPlayer, MediaPlayerError>> = LocalResource::new(MediaPlayer::new);

Now, however, I want to listen to error events on the HtmlAudioElement (or really, any event, error is just easy to test by pointing the source at a nonexistent file).

leptos_use features the use_event_listener function, which AIUI should create a signal that does that if I can provide it the correct inputs. This is where I get stuck - I don't understand how I'm supposed to hand this a signal that is not a T to this function.

I've attempted this:

impl MediaPlayer {
  pub fn audio_element(&self) -> EventTarget {
    self.audio_element.clone().into()
  }
}

# ---

let player: LocalResource<Result<MediaPlayer, MediaPlayerError>> = LocalResource::new(MediaPlayer::new);
let audio_element = move || {
  if let Some(Ok(p)) = player.get() {
    Some(p.audio_element())
  } else {
    None
  }
};

use_event_listener(audio_element, media_events::error, |ev| {
    logging::error!("{:?}", ev);
});

But this doesn't compile - it doesn't look like derived signals implement the necessary trait (nor do Memos, FWIW):

help: the trait `IntoElementMaybeSignalType<EventTarget, _>` is not implemented for closure

Doing this in an Effect works just fine, and is sufficient to get the log to show up, but that's not what I want to ultimately accomplish here:

Effect::new(move || {
  let audio_element = if let Some(Ok(p)) = player.get() {
    Some(p.audio_element())
  } else {
    None
  };

  use_event_listener(audio_element, media_events::error, |ev| {
    logging::error!("{:?}", ev);
  });
});

I have a feeling I'm going completely in the wrong direction here, but I don't even know where to begin looking for an alternative approach. I suppose I could implement these things with just web_sys, but then the state machine moves outside the scope of leptos and propagating events to the UI becomes cumbersome.

Anyone mind clearing up my confusion?

[gbj]

Is there any reason to use a LocalResource here rather than calling spawn_local to do the async work and then creating the rest of the UI? It doesn't look like it's reactive, and it doesn't look like you need to use it with Suspense etc.

Otherwise, adding an event listener is an effect, so calling an Effect to read from the LocalResource and check whether it's Some is reasonable.

[TLATER]

Trying to filter out the relevant content, simply for brevity:

#[derive(Clone)]
pub struct MediaPlayer {
    context: web_sys::AudioContext,
    audio_element: web_sys::HtmlAudioElement,
}

impl MediaPlayer {
    pub async fn new() -> Result<Self, MediaPlayerError> {
        let context = web_sys::AudioContext::new()?;
        let audio_element = web_sys::HtmlAudioElement::new_with_src(DEFAULT_MP3)?;
        let source_node = context.create_media_element_source(&audio_element)?;
        let gain_node = context.create_gain()?;
        let analyser_node = context.create_analyser()?;
        analyser_node.set_fft_size(2048);
        analyser_node.set_smoothing_time_constant(0.8);

        source_node.connect_with_audio_node(&analyser_node)?;
        source_node.connect_with_audio_node(&gain_node)?;
        gain_node.connect_with_audio_node(&context.destination())?;

        Ok(Self {
            context,
            audio_element,
        })
    }

    pub fn audio_element(&self) -> EventTarget {
        self.audio_element.clone().into()
    }

    pub fn get_title(&self) -> String {
        // Hardcoded for now...
        "Journey".to_owned()
    }
}

#[component]
fn Controls() -> impl IntoView {
    let player: LocalResource<Result<MediaPlayer, MediaPlayerError>> = LocalResource::new(MediaPlayer::new);

    Effect::new(move || {
        let audio_element = if let Some(Ok(p)) = player.get() {
            Some(p.audio_element())
        } else {
            None
        };

        use_event_listener(audio_element, ssr_safe::media_events::error, |ev| {
            logging::error!("{:?}", ev);
        });
    });

    view! {
      <div class="notification">
        <Suspense fallback=move || "Initializing audio player...">
          <ErrorBoundary fallback=|errors| { "Failed to initialize audio player" }>

            <div class="level is-mobile">
              <div class="level-left">
                // The play/pause/etc button
                <div class="level-item">
                  <button class="button" type="button">
                    <span class="icon is-medium" />
                  </button>
                </div>
              </div>

              // The title display
              <div class="level-item">
                {move || {
                  Ok::<_, MediaPlayerError>(player.get().transpose()?.map(|p| p.get_title()))
                }}
              </div>

              // The artist display
              <div class="level-right">
                <div class="level-item">Artist</div>
              </div>
            </div>

          </ErrorBoundary>
        </Suspense>
      </div>
    }
}

... but I'm just getting started with this. It's indeed pretty much just having an <audio> element, but without the element, and an audio context on top of it which has other events that need to be tracked. I'm basically just wrapping the Web Audio API.

That said, I think I'd have exactly the same issues if I had just an <audio> element?

[gbj]

So, in this example, none of the real logic really has anything to do with the framework: it starts with an effect (directly creating an element), threads it through a resource, and then reads it out in another effect. You can skip the intervening steps of the resource and the effect.

I would suggest something more like this, written in direct style by spawning an async task with spawn_local.

#[component]
fn Controls() -> impl IntoView {
    let (title, set_title) = signal(None::<String>);
    
    // if you're not using SSR, omit this cfg check
    if cfg!(feature = "hydrate") {
      spawn_local(async move {
        let Ok(player) = MediaPlayer::new().await else {
          todo!();
        }
        let element = p.audio_element();
        set_title.set(/* whatever */);
        // audio_element() should probably return HtmlAudioElement 
        // that way you can use this on() method from leptos::html::ElementExt
        // ElementExt is implemented for any T: AsRef<web_sys::Element>
        let evt_handle = element.on(html::events::error, /* ... */);
        // note that `evt_handle` will remove the listener when it's dropped; figure out some way
        // to handle the ownership so it doesn't do that, or just leak it
      });
    }

     // ...
     
       // The title display
       <div class="level-item">{title}</div> // simpler!

[TLATER]

I appreciate that! What if I now want to pause/resume from the UI, however? This leaves me with no handle on the player to access from the component, so I couldn't have a method like MediaPlayer::pause to call from an Action. Am I missing something obvious?

[gbj]

At that point it's just a question of storing it in a variable. In the example below I'm going to use StoredValue (which is like an Arc<_> but with the same Copy storage as a signal) and SendOption, which is convenient for storing !Send types that only exist in the browser.

    let (title, set_title) = signal(None::<String>);
    let audio_element = StoredValue::new(SendOption::new(None));
    
    // ...

      spawn_local(async move {
        let Ok(player) = MediaPlayer::new().await else {
          todo!();
        }
        let element = p.audio_element();
        audio_element.write_value().update(|n| *n = Some(element.clone()));
        
   /// somewhere else...
   let audio = audio_element.get_value();

Apologies for any errors, as I'm just typing this into GitHub, not the compiler.

[TLATER]

Apologies for any errors, as I'm just typing this into GitHub, not the compiler.

I did the same ;) But ta for the heads-up, will treat it as pseudocode.

I'll give this another shot with the new bits you've shown me, and report back with the full result once I'm done. Thanks so much for taking the time, these snippets are immensely helpful.