SRTStream delivers frames with out-of-order timestamps

[noho501]

Question

Hi,

I am testing SRT streaming using HaishinKit and I have a question regarding the tsbpd option.
I enabled tsbpd=1 on both the sender and the receiver in order to use Timestamp-based Packet Delivery mode (as the documentation suggests this should ensure packet delivery in presentation order).

However, when receiving CMSampleBuffer frames via StreamOutput on the caller side, the presentation timestamps often arrive out of order, which causes visible jitter.

Background / Tried Steps

Below are the URLs used:
Listener:

srt://192.168.2.39:9991?mode=listener&latency=500&streamid=myapp&rcvbuf=8388608&pkt_size=1316&tsbpd=1&sndbuf=8388608

Caller:

srt://192.168.2.39:9991?mode=caller&latency=500&streamid=myapp&rcvbuf=8388608&pkt_size=1316&tsbpd=1&sndbuf=8388608

This is the output code:

actor SrtStreamOutput: StreamOutput {
    nonisolated func stream(_ stream: some HaishinKit.StreamConvertible, didOutput audio: AVAudioBuffer, when: AVAudioTime) {
    }
    
    nonisolated func stream(_ stream: some HaishinKit.StreamConvertible, didOutput video: CMSampleBuffer) {
        guard let pixelBuffer = CMSampleBufferGetImageBuffer(video) else { return }
        let pts = video.presentationTimeStamp
        print("video ts", pts)
    }
}

Example log showing timestamps not arriving in increasing order:

video ts CMTime(value: 705812, ...)
video ts CMTime(value: 709056, ...)
video ts CMTime(value: 704508, ...)  // out of order
video ts CMTime(value: 707496, ...)
video ts CMTime(value: 712176, ...)
video ts CMTime(value: 715168, ...)
video ts CMTime(value: 710616, ...)  // out of order
video ts CMTime(value: 713608, ...)
video ts CMTime(value: 716728, ...)
video ts CMTime(value: 719332, ...)
video ts CMTime(value: 717772, ...)  // out of order
video ts CMTime(value: 722324, ...)
video ts CMTime(value: 720892, ...)  // out of order
video ts CMTime(value: 725316, ...)
video ts CMTime(value: 723756, ...)  // out of order
video ts CMTime(value: 726876, ...)
video ts CMTime(value: 734416, ...)
video ts CMTime(value: 731296, ...)  // out of order
video ts CMTime(value: 729736, ...)  // out of order
video ts CMTime(value: 732856, ...)
video ts CMTime(value: 740656, ...)
video ts CMTime(value: 737536, ...)  // out of order
video ts CMTime(value: 735976, ...)  // out of order
video ts CMTime(value: 739096, ...)
video ts CMTime(value: 743388, ...)
video ts CMTime(value: 746380, ...)
video ts CMTime(value: 741828, ...)  // out of order
video ts CMTime(value: 744948, ...)
video ts CMTime(value: 749500, ...)
video ts CMTime(value: 747940, ...)  // out of order
video ts CMTime(value: 752616, ...)
video ts CMTime(value: 751060, ...)  // out of order
video ts CMTime(value: 759904, ...)
video ts CMTime(value: 753792, ...)  // out of order
video ts CMTime(value: 755352, ...)
video ts CMTime(value: 756912, ...)
video ts CMTime(value: 762764, ...)
video ts CMTime(value: 765884, ...)
video ts CMTime(value: 761464, ...)  // out of order
video ts CMTime(value: 764324, ...)
video ts CMTime(value: 771868, ...)
video ts CMTime(value: 767444, ...)  // out of order
video ts CMTime(value: 769004, ...)
video ts CMTime(value: 770308, ...)
video ts CMTime(value: 778232, ...)
video ts CMTime(value: 775112, ...)  // out of order
video ts CMTime(value: 773428, ...)  // out of order
video ts CMTime(value: 776672, ...)
video ts CMTime(value: 781224, ...)
video ts CMTime(value: 779664, ...)  // out of order
video ts CMTime(value: 784344, ...)
video ts CMTime(value: 782784, ...)  // out of order
video ts CMTime(value: 791372, ...)
video ts CMTime(value: 785388, ...)  // out of order
video ts CMTime(value: 786820, ...)
video ts CMTime(value: 788380, ...)
video ts CMTime(value: 792932, ...)
video ts CMTime(value: 794492, ...)
video ts CMTime(value: 797352, ...)
video ts CMTime(value: 796052, ...)  // out of order
video ts CMTime(value: 803592, ...)
video ts CMTime(value: 798912, ...)  // out of order
video ts CMTime(value: 800472, ...)
video ts CMTime(value: 802032, ...)
video ts CMTime(value: 805152, ...)
video ts CMTime(value: 806712, ...)
video ts CMTime(value: 809444, ...)
video ts CMTime(value: 807884, ...)  // out of order
video ts CMTime(value: 815556, ...)
video ts CMTime(value: 811004, ...)  // out of order
video ts CMTime(value: 812436, ...)
video ts CMTime(value: 813996, ...)
video ts CMTime(value: 815556, ...)
video ts CMTime(value: 818676, ...)
video ts CMTime(value: 821408, ...)
video ts CMTime(value: 819848, ...)  // out of order
video ts CMTime(value: 829080, ...)
video ts CMTime(value: 824528, ...)  // out of order
video ts CMTime(value: 822968, ...)  // out of order
video ts CMTime(value: 825960, ...)
video ts CMTime(value: 835060, ...)
video ts CMTime(value: 831940, ...)  // out of order
video ts CMTime(value: 830380, ...)  // out of order
video ts CMTime(value: 833500, ...)
video ts CMTime(value: 836364, ...)
video ts CMTime(value: 837924, ...)
video ts CMTime(value: 839484, ...)

Environment

iPad mini 7 - iOS 26
iPhone 16 Pro - iOS 26
Haishinkit main branch

Is there a recommended or correct way to configure tsbpd within HaishinKit so that frames are delivered in presentation-time order?

Thank you for your guidance.

[noho501]

I found a way to resolve this issue, which might help others facing the same problem. Setting videoSetting.allowFrameReordering = false fixed it: the timestamps now increase consistently, and the jitter has been eliminated.

[shogo4405]

The TimeStamp mentioned here refers to something different from the CMSampleBuffer.presentationTimeStamp.
https://datatracker.ietf.org/doc/html/draft-sharabayko-srt-01#section-4.5

When using H.264 Main profile or higher, or when using H.265, B-frames become enabled.
In that case, the order of PTS (presentationTimeStamp) will match the result you observed.

Therefore, you will need to choose one of the two approaches:

1. Buffer a certain number of frames and sort CMSampleBuffers by their PTS.
2. Disable B-frames
   videoSettings.allowFrameReordering = false

Option 2 relies on the VideoToolbox feature.

[noho501]

Thank you very much for confirming; I truly appreciate it.