Avoiding transcoding ...

[bitmap]

Honestly, either is likely fine. They're both established codecs. If you want the best quality, avoid re-encoding lossy formats (this is why DTS-HD MA, FLAC, TrueHD, and other lossless codecs exist). If you need to increase compatibility and don't mind the artifacting, clipping, or other loss of quality, pick what user clients support. I would venture 768 kbps is more than enough. If you look at Opus, for example, around 160 kbps provides transparency in stereo audio -- double that for 5.1 and it's still great. Something like AAC I would want more like 256 kbps minimum for stereo and maybe 512 kbps for surround.

These are all subjective, though. They depend highly on your hearing ability (mine is quite damaged), client and personal setups (e.g., home theater system, sound bar, TV speakers), and client devices (e.g., Roku, Firestick, Jellyfin Media Player, web).

---

The Jellyfin client on iPadOS seems to transcode even AC3 to AAC.

The "Direct Streaming Info" shows "Audio codec: AAC", regardless if the track was AAC, AC3 or DTS.

So that should answer my question, it seems.

I should use AAC to avoid any transcoding.

So this is likely audio transcoding and remuxing. Not a big deal and does not trigger video transcoding, which is what you should be most worried about. Also forgot in my last post, make sure the audio is 5.1 channels or less (i.e., 5.1, stereo, dual/single mono).