Video Bitrate Calculation is inaccurate

[nickb937]

My setup is two houses with local gigabit wired networks, connected over the internet via IPSec VPN by fiber at 1000mbit/sec download, 100mbit/sec upload.  I have configured such that house A holds Jellyfin server and house B where a client runs is configured as a remote network.

Despite the reasonable upload speed, I often find that videos are marked as direct-play and experience a degree of playback stalling between the two houses.  Monitoring the network traffic on the router I can see that I should restrict streaming to 60mbit/sec and have configured the Jellyfin client as such.   Still all videos are streamed as direct-play and the videos will continue to stall.  In order to force transcoding of the video, I must set the maximum bandwidth in the client to a value less than recorded in the metadata for the video.

An example is Bill & Ted Face the Music, where I had to force the film to transcode by setting max bandwidth to 20mbit/sec.  Here bit-rate is reported as 32MBit/sec in Jellyfin (presumably the average bitrate) but the film actually peaks around 150Mbit/sec and even in the first few chunks the bitrate is higher than 32710kbit.  The problem is that if the reported bitrate of 32710kbit is used for the transcoding decision, then it's going to be the wrong choice for streaming this particular film.

Code:

Video
Title: 4K HEVC SDR
Codec: HEVC
AVC: No
Profile: Main 10
Level: 153
Resolution: 3840x1608
Aspect ratio: 2.40:1
Anamorphic: No
Interlaced: No
Framerate: 23.976025
Bitrate: 32710 kbps
Bit depth: 10 bit
Video range: SDR
Video range type: SDR
Colour space: bt709
Colour transfer: bt709
Colour primaries: bt709
Pixel format: yuv420p10le
Ref frames: 1

Code:

python -m ffmpeg_bitrate_stats "/tv/Films/Bill & Ted Face the Music (2020)/Bill & Ted Face the Music (2020) Bluray-2160p.mkv"
{
    "input_file": "/tv/Films/Bill & Ted Face the Music (2020)/Bill & Ted Face the Music (2020) Bluray-2160p.mkv",
    "stream_type": "video",
    "avg_fps": 24.39,
    "num_frames": 132121,
[b]    "avg_bitrate": 29690.019,[/b]
[b]    "avg_bitrate_over_chunks": 27318.108,[/b]
[b]    "max_bitrate": 150388.377,[/b]
    "min_bitrate": 64.551,
    "max_bitrate_factor": 5.065,
    "bitrate_per_chunk": [
        68.771,
        21202.516,
        44067.783,
        58371.812,
        53273.463,
        20805.554,
        17189.768,
        16395.582,
        9278.021,

Here we have a TV episode with a reported bitrate of around 15MBit, but a more detailed probe finds that the max-bitrate is nearly 30Mbit.

Code:

Video
Title: 4K HEVC SDR
Codec: HEVC
AVC: No
Profile: Main 10
Level: 153
Resolution: 3840x2160
Aspect ratio: 16:9
Anamorphic: No
Interlaced: No
Framerate: 23.976025
[b]Bitrate: 15565 kbps[/b]
Bit depth: 10 bit
Video range: SDR
Video range type: SDR
Pixel format: yuv420p10le
Ref frames: 1

Code:

$ python -m ffmpeg_bitrate_stats /tv/TV/What\ We\ Do\ in\ the\ Shadows/Season\ 5/What.We.Do.in.the.Shadows.S05E04.2160p.WEB.h265-EDITH.mkv  -p
{
    "input_file": "/tv/TV/What We Do in the Shadows/Season 5/What.We.Do.in.the.Shadows.S05E04.2160p.WEB.h265-EDITH.mkv",
    "stream_type": "video",
    "avg_fps": 24.39,
    "num_frames": 36377,
[b]    "avg_bitrate": 15572.731,[/b]
[b]    "avg_bitrate_over_chunks": 13650.258,[/b]
[b]    "max_bitrate": 29904.874,[/b]
    "min_bitrate": 48.298,
    "max_bitrate_factor": 1.92,
    "bitrate_per_chunk": [
        16903.486,
        14722.625,
        17009.637,
        16204.504,

This is what I'm using to calculate the bitrate: https://github.com/slhck/ffmpeg-bitrate-stats

It seems to me that with a slower, but more accurate probe of video bitrates, a better decision to transcode will be made and it should clear up a number of the reported issues we see on the forums around stalling/stuttering.

[TheDreadPirate]

If you have the know how, can you setup a librespeed docker container and speed test between a client on network B and the server on network A?

https://hub.docker.com/r/linuxserver/librespeed

[gnattu]

> The problem is that if the reported bitrate of 32710kbit is used for the transcoding decision, then it's going to be the wrong choice for streaming this particular film.

Not quite because networking also does not work the same way you think it is and I highly doubt your stalling is caused by those peaks because the avg could be way higher if the peak is that high. I really, really think it is that your ISP might doing some sort of QoS so that sustained upload is limited. You can verify that by just generating an 80Mbit constant bitrate video and see if that still stalls.

[nickb937]

I do have the know-how and I am aware of the max-data-transfer rate between the two networks and spent a good few years doing network engineering at an ISP too.

[nickb937]

The original post is not related to the ISP nor any rate-limiting that the ISP might be doing. The original post is stating a claim about the decision to transcode. I will restate here in a form that might be easier to understand:


The client specifies a max-bitrate of 40Mbit/sec.
The video states an average bit-rate of 32MBit/sec.
The decision is made not to transcode because the average bit-rate of the video 32Mbit < 40Mbit

The observable network data transfer varies between 5Mbit/sec and 150Mbit/sec for this 32Mbit/sec video, because of the encoding used (CRF) and more bits being allocated to scenes with a lot of change that are sufficiently brief on a 90 minute film to not push the average bit-rate particularly high.

The decision to transcode should be made based on the *peak* bitrate of the video, not the average bitrate. If the client specifies a max-bitrate of 40Mbit/sec, then the client should expect an observable data transfer that does not exceed 40Mbit/sec.

The probe of the film by ffmpeg_bitrate_stats shows the large skew between min-bitrate (6Mbit, avg ~28Mbit and max 150Mbit):

"avg_bitrate": 29690.019,
"avg_bitrate_over_chunks": 27318.108,
"max_bitrate": 150388.377,
"min_bitrate": 64.551,

And you can see a wide variance in required bandwidth in the first few frame-chunks:

"bitrate_per_chunk": [
68.771,
21202.516,
44067.783,
58371.812,
53273.463,
20805.554,
17189.768,
16395.582,
9278.021,

[nickb937]

> The problem is that if the reported bitrate of 32710kbit is used for the transcoding decision, then it's going to be the wrong choice for streaming this particular film.

Not quite because networking also does not work the same way you think it is and I highly doubt your stalling is caused by those peaks because the avg could be way higher if the peak is that high. I really, really think it is that your ISP might doing some sort of QoS so that sustained upload is limited. You can verify that by just generating an 80Mbit constant bitrate video and see if that still stalls.

The video contains 5285 chunks, the number of chunks of very high bitrate would of course raise the average-bit rate.  But if I set my client max bandwidth to 40Mbit/sec, then there are 640 chunks that will exceed that bandwidth and those are the chunks that will cause the stream to stall:

>>> len(list(filter(lambda x: x > 90000.0, data['bitrate_per_chunk'])))
5
>>> len(list(filter(lambda x: x > 80000.0, data['bitrate_per_chunk'])))
25
>>> len(list(filter(lambda x: x > 70000.0, data['bitrate_per_chunk'])))
58
>>> len(list(filter(lambda x: x > 60000.0, data['bitrate_per_chunk'])))
103
>>> len(list(filter(lambda x: x > 50000.0, data['bitrate_per_chunk'])))
234
>>> len(list(filter(lambda x: x > 40000.0, data['bitrate_per_chunk'])))
640
>>> len(list(filter(lambda x: x > 32000.0, data['bitrate_per_chunk'])))
1575

>>> len(list(filter(lambda x: x < 32000.0, data['bitrate_per_chunk'])))
3710
>>> len(list(filter(lambda x: x < 20000.0, data['bitrate_per_chunk'])))
1397
>>> len(list(filter(lambda x: x < 10000.0, data['bitrate_per_chunk'])))
344

[TheDreadPirate]

We get that. But theoretical performance, or performance of each network to an external speedtest site, is not the same as actual performance between the two networks. Setting up a librespeed container on one end would allow the you to test ACTUAL network performance between network A and network B.

[nickb937]

Your focus on network performance as the problem here is moot. The problem is one of choosing when to transcode and adhering correctly to the client specification: if the client specifies 40Mbit/sec, then the server should transmit at a maximum of 40Mbit/sec. Whether or not the network is capable of sustaining 40mbit/sec is a entirely separate problem, solvable by adjusting the client parameters. But since the server does not make the decision to transcode correctly, it is much harder to achieve the desired result. Speed-testing my network connectivity between two sites does not make this problem any less relevant than presented.