Splitting out the packaging repository

@nvllsvm and I both discussed and agreed that we should separate this out, as having it with the server backend doesn’t make much sense as the packaging framework grows and changes. For simplicity of discussion, let’s call the new repo jellyfin-packaging.

That said, I think it’s best to decide first how to handle combining the 3 repositories (or more, if we also add support for doing the various client apps within the same build framework) to build them, before doing the actual split. From my perspective there’s two main ways to proceed here:

  1. Submodule jellyfin-packaging into the main repo:

This would be “simple”, but also could have unintended side effects.

  1. Direct script-based cloning, where the main build script pulls the required repositories into a known location and goes the submodule initialization.

This might be more flexible, but would make the jellyfin-packaging repository the “main” one needed to build the others.

We also need to determine where to put things like e.g. the Dockerfiles - I understand @nvllsvm’s desire to keep them in the root of the main repo, however if we’re splitting out packaging that would imply that Docker packaging would be moved too. And this adds the complexity of new users determining exactly where they go to “run it”.

Soliciting feedback!

I feel it’s better to do in reverse and submodule jellyfin in jellyfin-packaging. This would require to restructure things in the build repo, but that’s acceptable. This way Dockerfiles could live in ., and, if submodules are pulled in correctly, all stuff would hum along happily.

Instead of having the scripts pull in repos, they should just use local clones. The repos are just too heavy to pull in, even if you do just shallow clones of the target branches.

Or they should at least support both options.

Frankly there is no real need to submodule anything IMO. But I just don’t like submodules. I’d rather see the jellyfin-web submodule removed as well and the packaging script take care of it all.

This would be a good idea because then we could also unify the build infra from the various Apps and Plugins into this repo as well. However I do somewhat agree with @EraYaN that submodules are a little dirty, especially since the main repo is still jellyfin and is the “central” one (unless we rename it jellyfin-server).

We can’t assume that anyone has everything they might want cloned when they start a build, or that they’ve organized their folders in a sensible way. Submodules do eliminate that issue.

We can rename the repo jellyfin-server (and use the cleaned up version for it) That won’t even mess peoples forks up. Maybe that is not such a bad idea.

But you can pull it in if people don’t have it, but when i have everything locally (or want to build from uncommitted code, that might be nice.

I’m not forcing anyone to submodules here, shallow clone should work just as well if build script is made properly.
My main point is that jellyfin-packaging should include jellyfin inside, but not the other way around.

Whatever we do, we need to leave the jellyfin/jellyfin repo as the home of the server source code. We’ve already had numerous rewrites of history so any more would serve to only frustrate.

However, I’m not sure I see much benefit by breaking things out right now. We one day might wish to remove the UI, but not right now. What exists currently serves us and is not broken.

1 Like

I agree, and there’s really nothing wrong with say, adding a plugin build facility into the main repo just for convenience, which is probably where I’ll put my WIP script.

I definitely think this is worthwhile long term, but not until, at the earliest, we get jellyfin-web fully independent from the server core.

I’d suggest the “monorepo” approach, like Google or Twitter use for their codebase. I think, it’s the perfect time to change to this for Jellyfin.

I’m a long time user and contributer of Contao-CMS, where this monorepo approach was made public last year. From the developing side it’s a great change, as it eases the workflow so much. (https://github.com/contao/monorepo-tools)

Babel is doing this as well, they have their own tool written (Lerna), that could be of use here:

Even if it wouldn’t be possibe right now, I’d say, now is the time to make a decission for the future in how to handle this.

Here is an interesting article about mono repos:

That’s why if the repo isn’t local you *then * force clone a copy of the repos needed to build that particular package

So I was thinking about this while I went about translating the Docker setup I wrote for jellyfin-android to the main Jellyfin packages, and ended up coming up with a fairly consistent and IMO reasonable solution.

We already have a build script in the main repo, and this pr makes sure the Android app uses the same formatting.

I think these scripts are fine, though a rewrite in Python would be beneficial. And I agree with @nvllsvm after more though that we should keep packaging configs for the main server in the jellyfin repository. This setup works perfectly, and once the separation is complete we could use the same idea for the jellyfin-web repository, building OS packages in the same format.

We can then have a global “manifest” repository, containing the jellyfin_manifest.json and build.py scripts from my current build setup, which already works fine for Plugins. This can then simply run the build.py (or install BASH via whatever OS mechanism and call the existsing BASH build scripts) from each repo to execute whatever build(s) we want to do.

As a whole, this seems like a very extensible and easy-to-use way for both the project to build official releases/packages/plugins, as well as for a user to build anything they want themselves, while also preserving the ability of each repository to be built on its own.