Investigate using CreateDIBSection

[jholveck]

Using CreateDIBSection bypasses the need for GetDIBits, similar to how we use XShmGetImage on Linux. It just writes the DIB straight into system-backed memory, and we can read it from there. This is faster, doesn't have as much GDI object churn, and also eliminates what is often a surprising sync point in GDI (GetDIBits).

I haven't really looked into this much, but it is probably worth investigating.

[BoboTiG]

I love the idea!

[halldorfannar]

This issue can be assigned to me. I have one question though; do we want to:

• Create a new backend (windows_dibsection.py) with a separate MSS class
• Modify windows.py to use CreateDIBSection insteade

I'm leaning towards the second option since CreateDIBSection is superior and doesn't have the fallback requirements that Linux has.

[BoboTiG]

If option 2 does not introduce issues, then lets go with it.

[halldorfannar]

Cool. I created a benchmark and did some experiments. This is what I found.

Both GetDIBits and CreateDIBSection are essentially at parity because BitBlt dominates the timing. The theoretical benefit of CreateDIBSection (eliminating the GetDIBits sync point) is masked by the much larger BitBlt cost.

To get higher frame rates
We'd need to bypass GDI entirely and use:

• DXGI Desktop Duplication API (Windows 8+) - Can capture at 60+ FPS
• Windows Graphics Capture API (Windows 10 1803+) - Modern capture API (cf Windows: support the new WindowsGraphicsCapture API #205)
  These APIs read directly from the GPU compositor without the CPU round-trip that BitBlt requires.

Now, I can still switch windows.py to use CreateDIBSection since it is architecturally a bit cleaner and more in line with XShm approach. But it's easy to argue that these changes are not worth it since they don't improve performance that much.

[halldorfannar]

I should probably experiment with the DXGI Desktop Duplication API next. The more modern Windows Graphics Capture API is an event-based API so would require some plumbing and blocking on my end to work synchronously.

[BoboTiG]

What about moving to CreateDIBSection in order to "complete" the GDI implementation, and provide a second backend with DXGI? If that sounds good on your end :)

[halldorfannar]

I will sleep on it! 😃

[jholveck]

Both GetDIBits and CreateDIBSection are essentially at parity because BitBlt dominates the timing. The theoretical benefit of CreateDIBSection (eliminating the GetDIBits sync point) is masked by the much larger BitBlt cost.

I'm actually a little surprised by this. My understanding was that BitBlt did an optional conversion from the DDB to a DIB, and stored it (possibly just as a blit, if possible) in your DIB. (I'm guessing that probably involves downloading a buffer from the GPU.) Then, the GetDIBits would blit again to the user-supplied buffer.

On the other hand, the CreateDIBSection would let you create a DIB destination for BitBlt that's already in a known part of user-space. So you wouldn't need the second blit that's part of GetDIBits.

This only surprises me a little bit, because I really don't know much about Windows. But was my basic understanding inaccurate or incomplete? The GDI sync point in GetDIBits takes so long that it overshadows the blits?

Also, just so I have my own basic sense, what's the resolution you used and approximate time you were seeing?

[halldorfannar]

All good points. I have many tests, at different resolutions. Let me add a few more to verify my theory that explains the behavior we are seeing.

[halldorfannar]

My theory:

DWM is throttling GDI screen capture to ~30 FPS (or half the display refresh rate). This is a fundamental limitation of the GDI->DWM path, not my implementation.
This explains why CreateDIBSection vs GetDIBits shows no difference - both hit the same DWM wall. The theoretical benefits of CreateDIBSection (eliminating one copy) are completely overshadowed by DWM frame pacing.

[halldorfannar]

Good news, on smaller screen captures we do see significant improvement using CreateDIBSection. Full screen is the same, due to some internal Windows shenanigans. This is an active scene that needs to be monitored closely 😆 I will keep you updated with my bulletins.

[halldorfannar]

Bummer, they both improved the same way. There was something going on with my Windows box that was causing the ~33 ms limit. That is now fortunately gone (rebooted this morning).

Region	Original (GetDIBits)	New (CreateDIBSection)
Full (2304x1536)	35.72ms (28 FPS)	34.49ms (29 FPS)
100x100	13.34ms (75 FPS)	14.67ms (68 FPS)
640x480	16.43ms (61 FPS)	14.18ms (71 FPS)

CreateDIBSection is architecturally cleaner so I support the suggestion of moving to it. I will clean up what I have and create a PR. It will include my new benchmark. Will come in handy as we add support for other more novel ways of doing this.