avc/h264 at greater than 4k

[exit]

I'm trying to play some VR files that have a resolution greater than 4096x4096.

The server has a 12th-generation Intel CPU for hardware acceleration. I've tried both QSV and VAAPI and both give me the error below. 

  Metadata:
    major_brand    : isom
    minor_version  : 512
    compatible_brands: isomiso2avc1mp41
    encoder        : Lavf58.12.100
  Duration: 00:13:07.71, start: 0.000000, bitrate: 30476 kb/s
  Stream #0:0[0x1](und): Video: h264 (High) (avc1 / 0x31637661), yuv420p(tv, bt709, progressive), 6000x3000 [SAR 1:1 DAR 2:1], 30139 kb/s, 59.94 fps, 59.94 tbr, 60k tbn (default)
    Metadata:
      handler_name    : VideoHandler
      vendor_id      : [0][0][0][0]
  Stream #0:1[0x2](und): Audio: aac (LC) (mp4a / 0x6134706D), 48000 Hz, stereo, fltp, 324 kb/s (default)
    Metadata:
      handler_name    : SoundHandler
      vendor_id      : [0][0][0][0]
Stream mapping:
  Stream #0:0 -> #0:0 (h264 (native) -> h264 (h264_vaapi))
  Stream #0:1 -> #0:1 (aac (native) -> aac (libfdk_aac))
Press [q] to stop, [?] for help
[h264 @ 0x5635c4050380] Hardware does not support image size 6000x3008 (constraints: width 0-4096 height 0-4096).
[h264 @ 0x5635c4050380] Failed setup for format vaapi: hwaccel initialisation returned error.
Impossible to convert between the formats supported by the filter 'Parsed_setparams_0' and the filter 'auto_scale_0'
Error reinitializing filters!
Failed to inject frame into filter network: Function not implemented
Error while processing the decoded data for stream #0:0
[libfdk_aac @ 0x5635c4052b00] 2 frames left in the queue on closing
Conversion failed!

With hardware acceleration the files play, but obviously, the CPU usage is off the charts. 

Is there a setting I need to change to allow greater than 4k encoding on jellyfin? 

Thanks 

[TheDreadPirate]

Negatory. It is a hardware limitation. Keep in mind that Quick Sync, NVENC, etc., are fixed function capabilities. You trade flexibility for speed. Also says it in the log.

Code:

[h264 @ 0x5635c4050380] Hardware does not support image size 6000x3008 (constraints: width 0-4096 height 0-4096).

[exit]

So my options are to resize the files or will a different format like h265 work?

Appreciate the reply!

[TheDreadPirate]

I think HEVC would work. The Quick Sync wiki says that Alder Lake supports 8K HEVC. Which that file is smaller than.

[exit]

Awesome thank you. I will post back if that works in case anyone else runs into this in the future