Any way to allow transcoded video bitrate to exceed original file?

[Homid Way]

I have a media library compressed with very efficient codecs and as soon as video transcoded(to place subtitles etc.) video quality is botched by my Vega 7 GPU which can't handle same video quality on the same bitrate as original file had. 

Basically now it goes like this:
Direct: 5 MB/s HEVC -> Good quality no problem
Transcoded: 5 MB/s HEVC -> Video is botched by inferior Amd encoder, has minor blocking and occasional jpeg artifacts

Is there any way to allow transcoding on higher bitrates to preserve as much video integrity as possible?
Now I only see options up to original file's bitrate, but for me it's fine if i would have to stream any bitrate during transcoding but keep quality as close to original file as possible

[TheDreadPirate]

The bit rate used is a fixed ratio. Depending on the original codec and trancoded codec, Jellyfin will adjust the bit rate accordingly. Going from HEVC to H264 will increase the bit rate by, IIRC, 40%. I guarantee you the issue is not the bit rate. AMD's H264 encoders were not, and still aren't, very good. None of their encoders for that era were good.

[Homid Way]

Issue is the bitrate.

I transcode HEVC -> HEVC so according to your post bitrate wouldn't change, but original file I encoded using NVENC and AMF just doesn't get the same quality on same bitrate.

But I tested AMF and with around 20-25% more bitrate does produce nearly same quality video

[TheDreadPirate]

Unfortunately, there is no way to force an increase of the bit rate of the transcode.