Support Bluetooth Audio as an audio source

[jeffreyabecker]

Is your feature request related to a problem? Please describe.
My wled hardware has a hard time picking up sound from the speakers in my bedroom because it lives inside my headboard.

Describe the solution you'd like
I would like to be able to pair my wled device with my phone as a bluetooth speaker and have that work as the source of data in audio_reactive as opposed to udp or a microphone

Describe alternatives you've considered
I'm currently running off (multiple) off-the-shelf wled controller modules so I cant really change the hardware. I guess I could get more hardware, set up the audio processing there and have it forward over udp.

Additional context
I'm willing to fork and submit a PR. (do you mind if I break the codebase up a bit if/when I do?)

[softhack007]

Additional context
I'm willing to fork and submit a PR. (do you mind if I break the codebase up a bit if/when I do?)

Hi @jeffreyabecker, contributions are always welcome :-)

Actually I'm skeptical if bluetooth audio input will work in the current architecture:

• esp32 bluetooth stacks are very big, and the firmware often does not fit into 4MB flash any more
• the current audiosource architecture is optimized around using the I2S driver (realtime sampling), i'm not sure if a bluetooth (or network) stream could be used instead.
• you can't leverage on the existing "UDP sound sync" feature - sound sync actually receives audio processing results (not audio samples) at ~50 results per second.

About "break the codebase up a bit" - honestly this is not a good time for refactoring. We are planning to convert the AR usermod into a "standalone" (off-tree) library, which can be linked into WLED-MM, upstream WLED or MoonLight the same way. This will need a lot of changes and refactoring. My fear is that additional refactoring in a PR will have lots of merge conflicts. The "standalone" (off-tree) library activities will happen in march/april, but after that I'll be happy to merge a bluetooth audio source PR from you.

[jeffreyabecker]

esp32 bluetooth stacks are very big, and the firmware often does not fit into 4MB flash any more

Thats deeply troubling tbh -- I wanted to make some bluetooth stuff eventually.

My fear is that additional refactoring in a PR will have lots of merge conflicts.

I'd absolutely agree with that, I'll check back on this in the summer. Just out of curiosity, are the audio-reactive effect implementations expected to move into that separate usermod?

the current audiosource architecture is optimized around using the I2S driver (realtime sampling), i'm not sure if a bluetooth (or network) stream could be used instead.
you can't leverage on the existing "UDP sound sync" feature - sound sync actually receives audio processing results (not audio samples) at ~50 results per second.

This was the primary reason for "break the codebase up a bit". My thinking was to:

• create an AudioState which holds refs to the globals and can be passed around
• create an abstract AudioProcessor which exposes a single void process(AudioState& state) method
• wrap the audio sync receive code into a UdpRecipientAudioProcessor class
• wrap the audio source architecture into a MicrophoneAudioProcessor class
• turn the loop function into something like processor->process(state)

[ewowi]

What could be feasible is to create a separate project / firmware which processes Bluetooth and send audio udp ( audio processing results (not audio samples)) over the network. So it's completely out of the WLED codebase. I agree with @softhack007 that Bluetooth is very heavy next to current code.
Could be an esp32 project but maybe even a smartphone app

[jeffreyabecker]

I'm definitely going to try this approach in the short term. Longer term, I'm not sure I'd be interested in maintaining entirely parallel development on something like that. If (once the module is out of tree) we could work towards componentizing the FFT analysis and UDP send/receive portions of the codebase, I'd be much more willing to take that on.