SOLVED: Inconsistent media playback behind oauth or basic auth

[urogna]

Hi everyone

I recently set up jellyfin for personal and family use.
I then added a caddy reverse proxy for certificate handling and an oauth reverse proxy (oauth2-proxy).
Everything runs and I can access jellyfin securly behind my google login (as well as my family) over my domain with https.
I also set up an endpoint that can only be accessed through my home IP for tv use (since to my knowledge there's no way to set up oauth in the TV app, or any native app).
The Problem:
On my android phone, some movies and shows dont play, I get: "Playback Error: The client isn't compatible with the media and the server isn't sending a compatible media format"
But this only happens when using my endpoint with oauth, and when I connect through my wifi and use the endpoint with the whitelisted IP, everything works fine.
I get the exact same behavior when setting up basic auth with the caddy reverse proxy.
I've dug into the problem quite a bit now, I even set up my own reverse proxy with oauth (with nextjs and next-auth) for debugging, and it looks like some calls don't reach the client because the secure environment gets stripped away from http headers and cookies, especially when the server want's to get info on how to transcode the media.
I get no errors from the server, so I guess the server really just sends the wrong transcoded media
I want to reiterate, every media plays perfectly fine when my reverse proxy directly communicates without any special cookies or https headers needed

Other devices work fine, on desktop every call reaches the client, ipad runs media fine, but some calls are blocked, I guess it just can play the delivered media... It looks like only android web has this problem.

Somehow I didn't find much on this issue... I'm stuck and could need some help 

Thank you guys for the awesome community and app

Calls that get blocked look like this, it retries 3 times, before the playback error alert, as you can see, my session isn't present at that time, only the csrf token and callback:

Code:

const nextUrl = {
  href: 'https://1ced1196db8f:3000/videos/ad9c6cb5-78a9-56ba-8eae-b6a4db46730c/master.m3u8?DeviceId=MY_DEVICE_ID&VideoCodec=h264&AudioCodec=aac&AudioStreamIndex=1&VideoBitrate=284361763&AudioBitrate=384000&AudioSampleRate=48000&MaxFramerate=23.976025&PlaySessionId=1a2d82b0a939485f9a03bdfab5bfcf41&api_key=24494f2b775d44caa5ea3fd1293e9900&TranscodingMaxAudioChannels=2&RequireAvc=false&Tag=8a2153562138c95610564ced75eaae19&SegmentContainer=ts&MinSegments=1&BreakOnNonKeyFrames=False&hevc-level=120&hevc-videobitdepth=8&hevc-profile=main&hevc-audiochannels=6&aac-profile=lc&TranscodeReasons=VideoCodecNotSupported%2C+AudioCodecNotSupported',
  origin: 'https://1ced1196db8f:3000',
  protocol: 'https:',
  username: '',
  password: '',
  host: '1ced1196db8f:3000',
  hostname: '1ced1196db8f',
  port: '3000',
  pathname: '/videos/ad9c6cb5-78a9-56ba-8eae-b6a4db46730c/master.m3u8',
  search: '?DeviceId=MY_DEVICE_ID&VideoCodec=h264&AudioCodec=aac&AudioStreamIndex=1&VideoBitrate=284361763&AudioBitrate=384000&AudioSampleRate=48000&MaxFramerate=23.976025&PlaySessionId=1a2d82b0a939485f9a03bdfab5bfcf41&api_key=24494f2b775d44caa5ea3fd1293e9900&TranscodingMaxAudioChannels=2&RequireAvc=false&Tag=8a2153562138c95610564ced75eaae19&SegmentContainer=ts&MinSegments=1&BreakOnNonKeyFrames=False&hevc-level=120&hevc-videobitdepth=8&hevc-profile=main&hevc-audiochannels=6&aac-profile=lc&TranscodeReasons=VideoCodecNotSupported%2C+AudioCodecNotSupported',
  searchParams: {},
  hash: ''
}

const headers = {
  ['Symbol(headers list)']: {
    cookies: null,
    ['Symbol(headers map)']: {
      'accept-encoding': { name: 'accept-encoding', value: 'gzip' },
      'cookie': {
        name: 'cookie',
        value: '__Host-next-auth.csrf-token=MY_CSRF_TOKEN; __Secure-next-auth.callback-url=https%3A%2F%2Fmy-domain'
      },
      'host': { name: 'host', value: 'my-domain' },
      'user-agent': {
        name: 'user-agent',
        value: 'Mozilla/5.0 (Linux; Android 10; K) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/123.0.0.0 Mobile Safari/537.36'
      },
      'x-forwarded-for': { name: 'x-forwarded-for', value: 'my-ip' },
      'x-forwarded-host': { name: 'x-forwarded-host', value: 'my-domain' },
      'x-forwarded-port': { name: 'x-forwarded-port', value: '3000' },
      'x-forwarded-proto': { name: 'x-forwarded-proto', value: 'https' }
    },
    ['Symbol(headers map sorted)']: [
      ['accept-encoding', 'gzip'],
      [
        'cookie',
        '__Host-next-auth.csrf-token=MY_CSRF_TOKEN; __Secure-next-auth.callback-url=https%3A%2F%2Fmy-domain'
      ],
      ['host', 'my-domain'],
      [
        'user-agent',
        'Mozilla/5.0 (Linux; Android 10; K) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/123.0.0.0 Mobile Safari/537.36'
      ],
      ['x-forwarded-for', 'my-ip'],
      ['x-forwarded-host', 'my-domain'],
      ['x-forwarded-port', '3000'],
      ['x-forwarded-proto', 'https']
    ]
  },
  ['Symbol(guard)']: 'request',
  ['Symbol(realm)']: {
    settingsObject: {
      baseUrl: undefined,
      origin: [Getter],
      policyContainer: { referrerPolicy: 'strict-origin-when-cross-origin' }
    }
  }
}
const cookies = { "__Host-next-auth.csrf-token": { "name": "__Host-next-auth.csrf-token", "value": "3ac9b306411e9db9dbce3d188413c445c1d2e58d1ae632271e72dffaf22a2c0e|9af1b39c4a92eef3e732571b7ace56bf543141f8ea2139aacbca3eacfed8acb9" }, "__Secure-next-auth.callback-url": { "name": "__Secure-next-auth.callback-url", "value": "https://my-domain" } }

[niels]

The Android apps do not support cookies. It might be possible cookies are stored in the webview but those are not shared with the video player.

[urogna]

The Android apps do not support cookies. It might be possible cookies are stored in the webview but those are not shared with the video player.

Thanks for the reply
I didn't clarify it well enough, I have this problem on android web browsers, but maybe on browser it has to do something with the video player as well, removing some cookies and headers.

[urogna]

The Android apps do not support cookies. It might be possible cookies are stored in the webview but those are not shared with the video player.

Thanks @niels this surely pointed me to the right direction.
I ended up letting traffic through the /Videos/* path, pass by my oauth proxy.