JSON schema/example for hwa.jellyfin.org

[BotBlake]

I messaged you on Matrix to point you in the right direction.
I hope that helps you find our dev room 😊

However I can already give some more detail about the features and issues behind pyTAB.
As jkboy already mentioned, it should be noted that pyTAB is still in development, therefore no release has been made yet.

It appears to be an error about something about the drm2 driver (pyTAB is obscuring the error as a generic ffmpeg error if I recall correctly).

This here is an issue we are currently fighting with. I dont know if you are experiencing the exact issue, but I am sure we will be abled to sort things out together in the Dev Chat at a later point. Generaly we currently have issues with the ffmpeg commands/ device selection - so a lot of current tests fail with weird reasons. We are on it, however the pyTAB crew and the hwa Server dev are currently pretty busy with other things 😅

I did notice some similarities when I was looking at the code (i.e: I had to force-print the ffmpeg_stderr to see why it was failing),plus I did some git archeology to understand which was born first, etc. Good to know!

If you are on the latest Version of pyTAB, it SHOULD create a seperate ffmpeg_err_log.txt file, that contains all Error Logs of ffmpeg.
Also pyTAB is trying to interpret the ffmpeg Error output and give a more detailed error reason itself. If the errors are too cryptic, it will default back to "generic_ffmpeg_error", hence your bad experience here.

I did like hwatest because by default looked at the ffmpeg installed version (whereas pyTAB gets a custom one from the Internet if I'm not mistaken). Doing so also helped me to notice that there's some input/output ffmpeg changes for 7.1 (I did notice pyTAB used ffmpeg 6.x) .
Plus, I'm not a big fan of full cloud/Internet dependency for running tools (e.g: downloading content / uploading results).

Now about some cool hidden features 😝
pyTAB could run fully local. It just expects SOMETHING to tell it what to do.
We just decided to not explicitly advertise that feature, as it would completely break compatibility with the hwa server.

Generaly pyTAB only needs the 3 informations to run:
1. a link to a ffmpeg binary to use
2. test files to use
3. commands to benchmark

All of theese things are provided by the hwa.jellyfin.org server by default.
(however the --server "URL" option could override it)

At the moment we use the exact same test files as hwatest and download them from the jellyfin repo:
https://repo.jellyfin.org/jellyfish/
When they already are present in the respective download folder, they will not automatically be downloaded again.

The ffmpeg binary we use also isnt "custom". We use jellyfin-ffmpeg and download from the jellyfin repo as well.
We only use the older ffmpeg release, because we didnt find the time to bump the versions yet. Also its totaly possible that a newer version MIGHT break our commands, so bumping the version could take some time.

We originaly wanted to make the output of pyTAB be interpretable without the server.
For now we just did not put our efforts into making it work.

However you will never HAVE TO upload your test results.
Also the Test results will NEVER contain personal Data.

Ah, that whitepaper it's gold. My first goal was to adapt JSON from hwatest to submit directly to the page. but if I'm to invest time, I'd rather do it on the proper tool, so I'm looking forward to keep the engagement on the helping!

Looking forward to work with you. Any collaboration is absolutely welcome.
Also glad if you found the tooling to be usefull or even potentialy usefull 😁

PS: All this because a friend asked about my device's performance for jellyfin to compare and I wanted to have an apple-apple comparison between devices. Hopefully, when I get this running we'll be able to speak the same language (perf. data).

I am realy curious: How did you even find this Project without being in our dev room? As far as I know, it was never specifically advertised yet, as its not even in Alpha.


Anyways, I hope we will hear each other soon on the Matrix 😆

PS: If you actualy want to compare an APPLE to an APPLE... Note that we are still working on apple support 😅
However we actualy HAVE put thought into how to support apple! - its on the ToDo List.

Oh I almost forgot 🗿

If you want to see what we are working on on pyTAB, you should defenetly take a look at our ToDo List:
https://github.com/users/BotBlake/projects/2