Other package management systems do one or two of the following things, but I haven't found any that can do all three at once, other than pkgsrc:
1) have good availability of pre-built binary packages for most platforms
2) facilitate building of packages directly from source, whether on non-mainstream platforms / architectures, and / or with or for the purpose of non-default package options
Having to use more than one package management system because one or the other doesn't do one of these things can be a real pain, and can make maintenance rather difficult.
- With both the community-maintained packages and the AUR packages, you can already find almost any binary that already exists
- If it doesn't exist, or if you need particular options/architectures, a PKGBUILD is basically a glorified Makefile with standard steps, so you can for example get binaries from a http source and use them directly, or pass exotic compilation flags during the build() phase
- It is of course possible to put dependencies inside a given package
- preparing a package is entirely done by your user, no need for root right. Inside the package you put your files in a fake root, and when the package is done you apply the package to your real fs. By default it is installed under /, so of course you need root rights, but you can install it under any root you desire. This is actually what's done when installing Arch: it basically runs pacman (the package manager) but instead of installing packages in the live environment's root, it installs packages in the mounted filesystem that will become your root after rebooting.
We are missing unprivileged builds and clean chroot builds for `makepkg`. Our interal devtools uses `systemd-nspawn` which requires root. But that should be easy to implement with the leftover code we tried to use with our reproducible builds tooling.
I love Nix, but this is true. The cross-platform support for pkgsrc is outstanding, and Nix's is still relatively limited.
Nix has better cross-compilation support is a little bit better than its native support, in terms of cross-platform coverage. But even then, I think, pkgsrc's platform coverage is in a different league. :)
It looks interesting. However, documentation is sparse. It doesn't clearly say what platforms are supported other than macOS and NixOS (which implies Linux in general, but doesn't clearly state it). Also, it seems to only target x86 and amd64. No mention is made of any other architecture, nor of any BSD other that macOS.
Very interesting, I didn't know pkgsrc could be used on Linux, I'll check it out. In general the only things keeping me from BSD are non free apps like Zoom so the next best thing is to BSDify my systems.
I'd try nixpkgs before trying pkgsrc if for some reason I wasn't satisfied with ones that come with the distro. Not sure why BSDfying packages is desirable by itself.
One reason to try it might be that pkgsrc mostly solves the problems Nix solves with conflicts in a much simpler way, by allowing multiple co-existing branches installed to different prefixes.
Nothing against nixpkgs, but if the user is curious about how things are done in BSD perhaps he should try xbps before nixpkgs. It was written by a former NetBSD committer who wrote some useful programs for NetBSD's userland. I am biased perhaps but I believe it is faster than any other Linux package manager. Always looking for something faster so I am welcome to being proven wrong on that.
As a pkgsrc committer, xbps is nothing like pkgsrc. The underlying stuff is much more similar to something like Arch or Alpine Linux than anything that exists in the BSD world, and it's much less configurable than pkgsrc.
Actual package recipes. BSD-style packaging systems are written in BSD make using a mk/ framework rather than Arch-style shell scripts. If a package already exists for FreeBSD Ports I would check that first because it's much more similar to what I'm used to.
Fundamentally pkgsrc is built on very low level tools, awk, shell, make, cwrappers, and pkg_install, although you can manage it with high-level ones (obviously pkgin is there, but there's also a lot of other third-party tooling).
Why are you comparing pkgsrc to pacman. The orginal comparison suggested in the parent comment was between xbps and nixpkgs or other Linux package managers.
pkgsrc boostraps itself using a program called "boostrap" in the pkgsrc directory. It builds a version of NetBSD's GCC toolchain and has no reliance on the host's userland. IME, this is very reliable and is its single greatest strength. Beyond that, pkgsrc is only as good as the build processes chosen by the author(s) of the software being built. These of course vary widely in sanity and depending on the packages one is building the otherwsise sane pkgsrc build process can quickly become a black box with packages that pull in many dependencies. It does break sometimes, but this is a fault of the target software authors, not pkgsrc. pkgsrc is first and foremost a system for building packages.
Linux distributions OTOH tend to be much more focused on binary package managers. Most Linux users do not build packages from source. There is really no comparison.
Your comments do not sound like those of a daily NetBSD user. I have been one for the last 15 years and am therefore all too familiar /usr/share/mk. Most Linux users do not seem very comfortable with BSD makefiles.
The original commented suggested that XBPS was similar to pkgsrc. I suggested that XBPS has far more in common with Arch Linux's package manager than pkgsrc or anything else in the BSD world. As you yourself note in your next paragraph...
> Your comments do not sound like those of a daily NetBSD user.
I have commit access. Since three years ago. My name is on the latest release announcement.
That's not nice and totally uncalled for. Three years is a very short time.
The original comment did not compare xbps to BSD package managers. What it said was that xbps was written by a former NetBSD developer and thus arguably has different qualities than other Linux package managers. It is a different approach from someone with different sensibilities than a Linux developer. I have read comments from Void Linux mantainers and users that say more or less the same thing.
I've been thinking of delegating mandatory crapware like zoom to my mobile phone. Needing to run stuff like that a few times a year is probably not worth preventing me from using an OS.
pkgsrc does work on Linux, but pkgin does not. I have no idea how to update my mirror and so forth using pkgsrc alone. Do not I need pkgin? I cannot do `pkg_add` at all.
I tried to make pkgin work, too, and I think it is possible to make it work with libbsd but I do not have the time. I wonder if anyone has done it.
I am the pkgin maintainer, and would be interested to hear about any issues you have building pkgin. It should build on any of the 23 platforms that pkgsrc supports, including Linux.
I immediately ran into an annoying issue, which is having the need to specify those directories manually. So I did, and the configure finished successfully when I ran:
actions.c: In function ‘verb_flag’:
actions.c:53:3: error: implicit declaration of function ‘strlcat’; did you mean ‘strncat’? [-Werror=implicit-function-declaration]
53 | strlcat(pkgtools_flags, "v", 1);
| ^~~~~~~
| strncat
Which tells me that I could fix it by providing `-I/usr/include/bsd/` and `-lbsd` or the like. I will give it a go. Any ideas anyway?
Edit: I added the above, and it went on to:
In function ‘strncpy’,
inlined from ‘read_repos’ at fsops.c:133:6:
/usr/include/x86_64-linux-gnu/bits/string_fortified.h:106:10: error: ‘__builtin_strncpy’ specified bound 8192 equals destination size [-Werror=stringop-truncation]
106 | return __builtin___strncpy_chk (__dest, __src, __len, __bos (__dest));
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
This does not sound too good, you probably want to fix it (along with the other warnings on Linux). In any case, I got rid of `-Werror` and I have a couple of undefined references. I will most likely get it to work, but you see my issue here, right? Is it the "official" way to compile it from source by the way?
Edit #2: I got a working binary by making those changes[1]. Last few lines:
CCLD pkgin
/usr/bin/ld: external/pkgin-var.o: in function `var_get':
/tmp/pkgin/external/var.c:99: warning: This function cannot be safely ported, use getline(3) instead, as it is supported by GNU and POSIX.1-2008.
make[1]: Leaving directory '/tmp/pkgin'
---
You might want to look into compiling it on Linux, because without those changes[1], you cannot do it. I was using commit e963d1e1a2409ed51d191d936b5f75e35bdf9197, version 20.12.1.
[1] You also have to include `<bsd/string.h>` for `strlcpy` on Linux. Only `actions.c` is affected.
---
If you want, I am willing to send you an e-mail with the diff, but I will fix the other warnings, too, then. Oh, and instead of relying on `libbsd`, one could just add `strlcpy` and the like themselves.
---
By the way, from where do I get the `repositories.conf` exactly? The supplied `repositories.conf` seems outdated. I tried using both `ftp://ftp.netbsd.org/pub/pkgsrc/packages/NetBSD/$arch/5.1/All` and `http://mirror-master.dragonflybsd.org/packages/$arch/DragonF...`, but they were not working.
Hmm, I tried to use a working one, but I got this:
processing remote summary (https://pkgsrc.joyent.com/packages/Linux/el7/trunk/x86_64/All)...
SSL support disabled
SSL support disabled
SSL support disabled
pkgin: Could not fetch https://pkgsrc.joyent.com/packages/Linux/el7/trunk/x86_64/All/pkg_summary.gz
I am currently using `file://` that points to the manually downloaded `pkg_summary.gz`. Not sure why SSL support is disabled when I configured it, it specifically asked for it, and I provided it through `--with-openssl=/usr/` and it passed because it found it.
I fixed it. I had to compile libfetch with `-DWITH_SSL'.
---
All in all, there are many problems that I ran into, but nothing that much of a big deal.
---
There is one problem I cannot seem to fix:
pkg_add: unable to verify signature: Signature key id 4735b9a256aaacaf not found
I keep getting this, and I have no clue how to change it or where it came from. Without this, nothing actually gets installed, so I cannot use pkgin or pkg_* until this is resolved. Do you have any clue? I tried to look for anything to change in pkg_install.conf or mk.conf but to no avail. Any ideas?
I'd been wondering how good pgksrc is on macOS for a little while, previously having used Homebrew wherever Nixpkgs was inconvenient or incomplete. Could pkgsrc could be a better choice?
pkgsrc is relatively nice for a cross-platform, source-based package manager in the old style, in that it provides convenient binary caches for most use cases.
I wonder: Has the writer of the post in the OP tried any next-gen source-based package managers, where the kind of isolation he describes as desirable between system and user packages is guaranteed between all packages altogether?
I'm always interested to try different things, but I don't understand why people seem to dislike Homebrew so much. I've been a mostly-happy user on both Mac and Linux for years.
Nix is probably the slowest and is missing many packages. It's Kool-Aid for the cool kids. MacPorts was good until everyone started using Homebrew. Homebrew has some weird quirks that drive me crazy, but it mostly works and is up to date. I tried pkgsrc and liked it, but a few common packages were weirdly named and it didn't have some Mac specific stuff.
Some new package manager that's a mix of pkgsrc, brew, and MacPorts would be awesome. Especially if portable and could somehow also replace nvm, rbenv, pythonenv, and things of that nature.
Nix evaluation can be slow sometimes, but installation is generally relatively fast, since everything in Nix is fully parallelizable. It's generally several times faster than apt, for installs that involve multiple large downloads, for example, and historically it's also faster than pacman.[1] (I think this is probably no longer the case with Pacman 6.0, which finally parallelizes downloads. In that case Nix evaluation time probably dominates.)
I imagine what you're identifying as slow is searching through Nix packages. On the old/stable release of Nix, you use something like `nix-env -qaP` for this, and that requires evaluation of every package in Nix, and can take a long time. `nix search`, however, is extremely fast on the unstable releases of Nix.
A Nix search using the old method is extremely slow, > 7 seconds:
I ~ time nix-env -f '<nixpkgs>' -qaP libreoffice
libreoffice libreoffice-7.0.4.2
libreoffice-still libreoffice-7.0.4.2
libreoffice-still-unwrapped libreoffice-7.0.4.2
libreoffice-fresh libreoffice-7.1.2.2
libreoffice-fresh-unwrapped libreoffice-7.1.2.2
libreoffice-qt libreoffice-7.1.2.2
________________________________________________________
Executed in 7.31 secs fish external
usr time 6.79 secs 335.00 micros 6.79 secs
sys time 0.52 secs 67.00 micros 0.52 secs
A Nix search using the new method, which leverages cached evaluation of Nixpkgs, takes about half a second:
nix-shell -p nixUnstable --run fish
time nix search --experimental-features 'nix-command flakes' nixpkgs libreoffice
<snip>
* legacyPackages.x86_64-linux.libreoffice (7.0.4.2)
Comprehensive, professional-quality productivity suite, a variant of openoffice.org
<snip>
________________________________________________________
Executed in 576.30 millis fish external
usr time 561.00 millis 252.00 micros 560.75 millis
sys time 16.07 millis 55.00 micros 16.01 millis
And installation, in either case, is pretty fast. It takes a few second to install, for example, Libreoffice on my system, with the vast majority of the time taken up being the download:
time nix-env -f '<nixpkgs>' -iA libreoffice
installing 'libreoffice-7.0.4.2'
these paths will be fetched (153.59 MiB download, 574.57 MiB unpacked):
<snip>
copying path '/nix/store/8yq9h91bsbgi62dhvh4xfqxvhq58b5y4-libreoffice-7.0.4.2' from 'https://cache.nixos.org'...
building '/nix/store/nhmf4ak414lrzy3387kbsx8brm8hckds-user-environment.drv'...
created 68 symlinks in user environment
________________________________________________________
Executed in 21.32 secs fish external
usr time 1.34 secs 0.00 micros 1.34 secs
sys time 0.43 secs 521.00 micros 0.42 secs
(I tried for a bit to get Linuxbrew installed so I could test the same functionality with it, but Homebrew isn't portable enough to work on NixOS, since it hardcodes a ton of command and interpreter paths to files that don't exist on some distros.)
tl;dr: Homebrew actually takes several times as long to install packages as Nix does, and uses up way more space despite a less isolated toolchain.
Now that I'm on my Ubuntu system, I can easily test Nix and Homebrew side-by-side.
To make it fair, I've removed almost everything I have installed with Nix, as well as Homebrew. Interestingly, Homebrew is using ~2x as much disk space, and listing my installed packages (none) with Homebrew takes ~30x as long as with Nix:
~
du -hsxc /nix/store /home/linuxbrew
238M /nix/store
584M /home/linuxbrew
822M total
~
time nix profile list
________________________________________________________
Executed in 19.88 millis fish external
usr time 11.21 millis 174.00 micros 11.04 millis
sys time 11.18 millis 141.00 micros 11.04 millis
~
time sudo -H nix profile list
0 flake:nixpkgs#legacyPackages.x86_64-linux.nixFlakes github:NixOS/nixpkgs/b2f2d04f01b3de3abad0dd9bc48ad7a3ee32bbc6#legacyPackages.x86_64-linux.nixFlakes /nix/store/g4xva7x46h5bqqh811n12dzjc7325x21-nix-2.4pre20210326_dd77f71
1 flake:nixpkgs#legacyPackages.x86_64-linux.starship github:NixOS/nixpkgs/b2f2d04f01b3de3abad0dd9bc48ad7a3ee32bbc6#legacyPackages.x86_64-linux.starship /nix/store/5n36vabqg701l9nji30m1r004zcy4q0w-starship-0.51.0
________________________________________________________
Executed in 29.91 millis fish external
usr time 27.30 millis 380.00 micros 26.92 millis
sys time 4.72 millis 283.00 micros 4.43 millis
~
time brew list
==> Formulae
________________________________________________________
Executed in 881.81 millis fish external
usr time 527.24 millis 411.00 micros 526.83 millis
sys time 173.15 millis 298.00 micros 172.85 millis
Homebrew takes about twice as long to search for packages related to Libreoffice (results omitted to HN won't tell me this post is too long):
~
time nix search nixpkgs libreoffice >/dev/null
________________________________________________________
Executed in 554.37 millis fish external
usr time 537.39 millis 434.00 micros 536.95 millis
sys time 16.44 millis 289.00 micros 16.15 millis
~
time brew search libreoffice >/dev/null
________________________________________________________
Executed in 1.33 secs fish external
usr time 578.16 millis 0.00 micros 578.16 millis
sys time 124.57 millis 367.00 micros 124.20 millis
I had some trouble looking for large packages to install with Homebrew and Nix for comparison. It seems that GUI apps (Libreoffice, Firefox, etc.) tend to be packaged exclusively as casks for Homebrew, which means they don't exist on Linux. So instead, I selected a handful of packages from the Linux formulas list[1] that I figured would pull in a substantive dependency closure. Apparently there's no bottle for Crystal, so it took a very long time to install because it had to compile from source. Prior to that, though, installation crashed because of a ulimits issue when ‘pouring the bottle’ for LLVM:
~
time brew install imagemagick bat feh crystal python mruby
==> Downloading https://ghcr.io/v2/linuxbrew/core/zlib/manifests/1.2.11
######################################################################## 100.0%
==> Downloading https://ghcr.io/v2/linuxbrew/core/zlib/blobs/sha256:6f604280b9e056da163b81cef3076b2e39bbd2b8d3cc8c36d27c19c472c760a4
==> Downloading from https://pkg-containers.githubusercontent.com/ghcr1/blobs/sha256:6f604280b9e056da163b81cef3076b2e39bbd2b8d3cc8c36d27c19c472c760a4?se=20
######################################################################## 100.0%
==> Downloading https://ghcr.io/v2/linuxbrew/core/libpng/manifests/1.6.37
######################################################################## 100.0%
==> Downloading https://ghcr.io/v2/linuxbrew/core/libpng/blobs/sha256:c5d219b27780c730af50693110ed2c79e50200d3595c7e395998fe3950354ec9
==> Downloading from https://pkg-containers.githubusercontent.com/ghcr1/blobs/sha256:c5d219b27780c730af50693110ed2c79e50200d3595c7e395998fe3950354ec9?se=20
######################################################################## 100.0%
==> Downloading https://ghcr.io/v2/linuxbrew/core/bzip2/manifests/1.0.8-1
######################################################################## 100.0%
<snip>
==> Installing crystal dependency: llvm@9
==> Pouring llvm@9--9.0.1_5.x86_64_linux.bottle.tar.gz
Error: Too many open files @ rb_sysopen - /home/linuxbrew/.linuxbrew/Cellar/llvm@9/9.0.1_5/bin/llvm-jitlink
________________________________________________________
Executed in 600.02 secs fish external
usr time 203.24 secs 479.00 micros 203.24 secs
sys time 70.80 secs 313.00 micros 70.80 secs
I decided to re-run the brew install command, but skip Crystal, which only took a few more seconds:
~
time brew install imagemagick bat feh python mruby
<snip>
==> Installing mruby
==> Pouring mruby--3.0.0.x86_64_linux.bottle.tar.gz
/home/linuxbrew/.linuxbrew/Cellar/mruby/3.0.0: 323 files, 14.7MB
________________________________________________________
Executed in 20.18 secs fish external
usr time 9.56 secs 0.00 micros 9.56 secs
sys time 3.06 secs 761.00 micros 3.06 secs
I left Crystal in when I tested Nix, since I knew it would be no problem, and Nix was done in ~36 seconds, meaning Homebrew ~17x the time, skipping one package entirely.
After the two install operations, Homebrew was using several more gigabytes of storage than Nix:
du -hsxc /nix/store /home/linuxbrew
2.0G /nix/store
4.9G /home/linuxbrew
6.8G total
I'll reply to this comment with a ‘warmed-up’ test of installing a single additional package, as well. But it seems pretty clear already that it's misleading at best to characterize Homebrew as faster than Nix.
Now for a single-package ‘warmed up’ test, and some cleanup on both sides to see what the total installed size looks like after autoremoval of orphaned dependencies, build-time dependencies, etc.
Installing one package with Homebrew, where the whole dependency closure is bottled, no building from source:
~
time nix profile install nixpkgs#graphicsmagick
________________________________________________________
Executed in 11.58 secs fish external
usr time 701.80 millis 500.00 micros 701.30 millis
sys time 310.06 millis 319.00 micros 309.74 millis
So in terms of time, they're not really in the same ballpark. Nix is operating in ‘wait a few seconds’ time, and Homebrew is operating in ‘go make yourself a cup of coffee and come back’ time.
And after running `brew cleanup` and `brew autoremove`, then `nix-collect-garbage` and `nix-store --optimize`:
~
du -hsxc /nix/store /home/linuxbrew
1.9G /nix/store
3.9G /home/linuxbrew
5.8G total
Nix's output is a mess, visually, and that'll change in the future. But note that it doesn't bother me with the whole dependency closure, and instead tracks what I actually wanted to install.
Finally, a removal test. I'll uninstall everything but graphicsmagick, with both package managers.
time brew remove imagemagick bat feh python mruby
Error: Refusing to uninstall /home/linuxbrew/.linuxbrew/Cellar/python@3.9/3.9.5
because it is required by freeglut, glib, graphicsmagick, jasper, libheif, liblqr, llvm, mesa, mesa-glu and shared-mime-info, which are currently installed.
You can override this and force removal with:
brew uninstall --ignore-dependencies imagemagick bat feh python mruby
Oops. A second try, since Homebrew put me in dependency hell because it doesn't know the difference between what's installed as a dependency and what I explicitly asked it to install (I thought this functionality was needed to implement `brew autoremove`??).
~
time brew remove --ignore-dependencies imagemagick bat feh python mruby
Uninstalling /home/linuxbrew/.linuxbrew/Cellar/imagemagick/7.0.11-13_2... (805 files, 31.4MB)
Uninstalling /home/linuxbrew/.linuxbrew/Cellar/bat/0.18.1... (12 files, 5.5MB)
Uninstalling /home/linuxbrew/.linuxbrew/Cellar/feh/3.7... (26 files, 716.9KB)
Uninstalling /home/linuxbrew/.linuxbrew/Cellar/python@3.9/3.9.5... (2,650 files, 76.9MB)
Uninstalling /home/linuxbrew/.linuxbrew/Cellar/mruby/3.0.0... (323 files, 14.7MB)
________________________________________________________
Executed in 1.11 secs fish external
usr time 874.06 millis 420.00 micros 873.64 millis
sys time 224.91 millis 261.00 micros 224.65 millis
~
time brew autoremove
<snip>
Uninstalling /home/linuxbrew/.linuxbrew/Cellar/libgpg-error/1.42... (48 files, 1.8MB)
________________________________________________________
Executed in 3.35 secs fish external
usr time 2.51 secs 424.00 micros 2.51 secs
sys time 0.82 secs 261.00 micros 0.82 secs
Nice and fast. :)
Now for Nix:
~
time begin
nix profile remove '.\*'
nix profile install nixpkgs#graphicsmagick
end
________________________________________________________
Executed in 2.99 secs fish external
usr time 499.04 millis 871.00 micros 498.17 millis
sys time 126.67 millis 527.00 micros 126.14 millis
Not as fast, because I garbage-collected the evaluation of Nixpkgs itself, so it fetched and re-evaluated that. (Also I did this in a weird way because the UI for the `nix profile` command is incomplete and still kinda clunky.)
I also need to clean up with Nix:
~
time nix-collect-garbage -d
removing old generations of profile /nix/var/nix/profiles/per-user/pxc/profile
removing generation 49
<snip>
deleting '/nix/store/znvl3v02hy8ffkhwc6m9z1q5pj8j6ppj-setup-hook.sh'
deleting '/nix/store/trash'
deleting unused links...
note: currently hard linking saves -1.14 MiB
188 store paths deleted, 1574.20 MiB freed
________________________________________________________
Executed in 1.23 secs fish external
usr time 14.30 millis 160.00 micros 14.14 millis
sys time 10.70 millis 96.00 micros 10.60 millis
And after cleanup, the usage:
~
du -hsxc /nix/store /home/linuxbrew
412M /nix/store
3.2G /home/linuxbrew
3.6G total
Is there a cleanup command I've missed for Homebrew?
Performance considerations aren't really among my main concerns when it comes to package management, so I didn't mention them in my original post about what's ‘wrong’ with Homebrew. But I'd have to add it now.
I don't even like Homebrew, but I use it because of the network effect. Several years ago I tried a bunch of package managers in identical VMs on MacOS and noticed Nix was slower than all the others (all operations), missing packages, and some packages were broken. During that experiment I really liked pkgsrc, Homebrew didn't have a few things I needed and outright refused to install TeX Live (it literally said it was too hard to build). I stuck with MacPorts for a few years until MacPorts started having many broken packages from lack of maintainers, then I reluctantly switched to Homebrew.
> pkgsrc doesn’t have Apple M1 binary packages available yet.
basing this reply on my past experience with pkgsrc (in a netbsd or linux environments though) as long as you can point pkgsrc to a decent compiler and it can complete its bootstrap phase, it'll likely work.
Homebrew's biggest issue is that it isn't a package manager for MacOS. You can interpret that however you want, but it's versatility is extremely limited compared to "full-fat" package managers like apt and pacman. At the end of the day, all of the MacOS package managers fit together to perform separate stop-gap solutions, where none of them really fell that great to use in the first place. Hell, pacman is so good that it's one of the core reasons I'll probably never end up fully switching my workflow to MacOS.
Pacman and friends used to work on macOS (source: I compiled and used them) and maybe they still do, but when it moved to bash 4 while apple is shunning newer versions, it became too much work to build all of the prerequisites.
In addition to the valid criticisms in sibling comments:
I don’t like that it changes the rules on file ownership in /opt and /usr/local.
And it’s petty but I really don’t appreciate the beer brewing metaphor. It doesn’t fit and it feels forced. Giving the project a searchable name is fine but cutesy names are irritating when I have real work to do. Just name the tool and especially commands for what they do.
Yes, the beer brewing stuff, formulae, taps, kegs, casks, and bottles stuff is annoying. I have no idea what "keg only" is supposed to mean or why I see that phrase sometimes.
brew cask is a whole separate quasi-package management system built around pre-built macOS GUI applications distributed/installed as .app folders, same as you'd get downloading a .dmg from the web them doing a drag and drop install.
Think of it as an installer fetcher and runner, as opposed to a collection of integrated and uniformly built packages.
It's great, because downloading and running installers manually by clicking around sucks. But it's totally separate from the rest of Homebrew, which is a monorepo of build recipes targeting a separate base system, similar in that basic way to things like pkgsrc and FreeBSD Ports.
I also hate the brewing metaphors. I don't know why MacOS developers feel so compelled to abstract integral parts of your program away into unrecognizable metaphors. It's like asking for ketchup at the counter of a diner, and they insist that they only have "Heinz Tomato Sauce". Nobody in real life does this because nobody likes a smartass, so it completely escapes me why people think it's suddenly okay to do in software, particularly software designed to be used in a professional, power-user workflow.
It's more like using a plumber analogy to make sandwiches. I mean plumbers eat lunch right? So we can refer to condiments as grout and bread and meat as tiles and pipes are the sandwiches because plumbing.
Also since this is in the context of making a sandwich a bunch of options to 'plumber' mention things like toaster oven wattage and temperature.
set PLUMBER_PREHEAT=350
set PLUMBER_BACKEND=toaster
This is all kind of a pain. Let's use pipes. Except for some reason whoever defined this uses thousand island instead of russian dressing and expects the sandwich on a plate.
It's got a larger user and contributor base on macOS than competing package managers, and that works in its favor, especially in terms of package selection. I'm not a total hater; there are people I respect who prefer Homebrew over tools that I like better for reasons I understand. And package management is a peculiar interest of mine, so I care a lot about details and there isn't any package management system that totally satisfies me.
That said, every time I've used it, I've been struck by design flaws that make Homebrew feel incomplete or immature compared to most Linux system package managers. Some examples:
• The last time I used it, Homebrew didn't even support recursive uninstallation of orphaned dependencies, guaranteeing the accrual of entire packages of cruft over time. This was apparently finally fixed (in the crummy way that apt-get, the worst of all the Linux system package management tools, handles it) only in 2020 [1].
• A secondary package manager taking over all of /usr/local is quite presumptous and not portable at all
• Homebrew's combination of an assumption of a single-user setup with the choice of installing to global directories in the FHS is awful, and imo it's straight up deceptive to claim that Homebrew's default setup meaningfully supports ‘unprivileged package installation’
• `brew cask` is useful, but the cleavage between it and the rest of Homebrew is awkward, and it's more like an installer-fetcher à la Chocolatey than a real package management system
• Homebrew's packages are built with a relatively low degree of isolation from the base system's libraries and toolchain, so it's not unusual for a macOS upgrade to break installed Homebrew packages, and the build outputs and bottles associated with a package are given per-macOS release. (To be fair, the former is a pretty common problem for all kinds of software on macOS, as well.)
• I really don't like Linuxbrew's abuse of /home, which you have to stick with if you want to use bottles (binary caches)
• Homebrew just doesn't support managing dependencies on libraries in some languages (e.g., Perl) at all [2]— even though it includes packages that depend on them— which fails to meet my bare minimum expectations for a general-purpose package manager. In practice, this means that you have to install additional, language-specific package managers to use Homebrew; you have to manage dependencies in them yourself; and Homebrew packages require you to install these dependencies in system-wide locations, polluting your whole operating system (a potential source of dependency hell, as well as a practical nuisance) for their sake. The latter also, of course, undercuts the unprivileged installation claim that Homebrew touts.
To be fair to Homebrew, it's been a while since I used a Mac on a daily basis, and I expect that Homebrew has improved in some ways in the meantime. I know for certain that it's improved, to reiterate an example given earlier, with respect to some, if not all (i.e., not those managed externally) orphaned dependencies. And Homebrew also has some strengths, namely its very slick and orderly command-line interface.
But the real reasons I try to use Homebrew as little as possible aren't about Homebrew being _bad_ so much as my preference for tools with other strengths (which require a design different from Homebrew's to implement). On macOS, I try to use Nixpkgs and nix-darwin as much as possible because I prefer things like:
• declarative configuration rather than imperative state management
• fully isolated builds, more predictable solutions
• using a single package management tool for libraries and apps of all languages
• building .apps from source like any other package
• atomicity of upgrades
• freedom to roll back configurations/installations at any time
• ability to mix and match packages from different releases, modify dependencies for one project without affecting another
• never installing language interpreters or libraries globally, except for login shells
I don't really want to use anything that doesn't have those features, and for that on macOS, Nix is the only game in town. Homebrew is a pile of state to manage, and I'd rather have intentions to declare than a pile of state to manage. That issue isn't unique to Homebrew, of course.
Pkgsrc doesn't share some of Homebrew's (in my view) other defects, like its abuse of the FHS on Linux, single-user design/bad permissions configuration, building many packages directly on top of macOS system libraries that change from OS release to OS release when it doesn't have to, or the expectation that library dependencies for some languages will be managed manually/externally (and in global directories, to top it off), so that's what makes Pkgsrc interesting to me as an alternative to Homebrew, even though Nix will still always be my first-line choice on macOS.
You mean for installation on macOS? For current versions of Nix and macOS, you just invoke the installer with an additional command-line flag at installation time, per the manual[1]:
sh <(curl -L https://nixos.org/nix/install) --darwin-use-unencrypted-nix-store-volume
I don't remember whether I actually did this on my system. I think I manually created an additional volume for /nix, but encrypted it. That approach is also now described in the manual, should you prefer it.[2]
(Unencrypted should be fine anyway, as the Nix store is designed/assumed to be public (world-readable). Packages and tools in the ecosystem never put secrets there.)
The two that I know of currently are Nix and Guix. Both target fewer platforms than Pkgsrc, so we know right out of the gate that they're not suitable for all Pkgsrc use cases.
The question of what they depend on needs fleshing out before it can be answered. Both Nix and Guix package software in a way that is, in general, hermetic. This means that they don't assume the installation of anything outside their domain— they don't require you to have anything installed on the base system to get going with them. They don't use your system's shell and they don't use your system's C library. As an exception on macOS, Nixpkgs allows some impurities for some macOS packages, e.g., they are allowed to call xcodebuild. But in general, they include their own toolchains in their entireties. So if what you're getting at is ‘where do they depend on the base system during operation’, the answer is, more or less ‘nowhere’.
If the real thrust of your question is rather ‘what do you need to build them from source’, the answer, for now, in both cases is ‘a bunch of shit that they kindly prepackage for you in a binary tarball, if you want to bootstrap your own Nix or Guix on a given platform’. Guix has been doing a ton of work to reduce the size of their bootstrap seed, and it seems like within a few years they'll have it down to a hex monitor from which they can build a series of Scheme interpreters and C compilers until they have the whole system.
If the main thing you're asking about is ‘what do they use internally to process their package sets’ or something like ‘what's the minimum installed size of these package managers’: for Nix/Nixpkgs you need the Nix language interpreter (written in C++), plus at least what's packaged in the ‘standard environment’ for Nixpkgs (GNU bash and coreutils, plus a libc and a C compiler which vary by platform); and for Guix you need Guile, an implementation (interpreter and runtime) of Scheme. I've never worried about their installed sizes, since they're much smaller than the total size of everything you might want to install with them.
Idk the size of the full dependency closure for Guix and I don't have a great way to look it up at the moment. In the Nix case, the binary installation tarball is ~17 MB compressed and ~72 MB uncompressed, when you install it.[1]
Happily, if Pkgsrc is easier to get started with due to its smaller footprint, it looks like in the future users will be able to use Pkgsrc to install Nix.[2] :)
(For my part, I wouldn't mind if Nixpkgs included Pkgsrc, too. I wish it included Guix.)
I really like pkgsrc. I'm not really qualified to give a technical justification for it. It just had the right simple, clean feel to it, which NetBSD in general has. I would even go so far as to say that it was fun to use.
I consequently once spent quite some time trying to get it working under Cygwin so I could use it at work, but never got past problems with Cygwin group names that have spaces in them. It was too complex for me to fix....
No mention of DragonflyBSD? They used pkgsrc as their main package manager for 10 years, portable or not, it still required quite a bit of maintenance. Seems like it was too much in the end, second-class citizen rights are not great.
it was a big help in Solaris to VWware migrations I did circa 2008-2012. I had one bash script that would install the whole site on either, bootstrapping almost everything from pkgsrc.
I wonder if anyone is going to create a linux distribution with a base system baked in with pkgsrc managing the applications. It would be very useful for containers and WSL.
I used it with Netbsd 1 and it was the first thing to install in Solaris.
Other question: how does one handle detection whether upgrades exist?
Something like renovatebot but using pkgsrc.
Or what is the best way in 2021 to trigger fresh builds against the latest packages?
At some point we will transition the repository of record to something else, but it's being done carefully. We want to do it right, and there aren't many volunteers willing help with the work.
No, I was just surprised - I don't know anything about NetBSD (nor pkgsrc for that matter), but I worked with FreeBSD some 15 years ago and there were plans to move to SVN even back then. A quick search told me that the switch was completed in 2008 (a little bit after I left, begrudgingly, for Linux).
What challenges are there in switching to another VCS? Is the difficulty coming from the need to preserve history, or the need to re-evaluate workflows around the repo, or something else entirely?
Not for prebuilt binaries (variants are just separate packages) but because the pkgsrc repo is just a collection of Makefiles, you can cd into the directory and run
1) have good availability of pre-built binary packages for most platforms
2) facilitate building of packages directly from source, whether on non-mainstream platforms / architectures, and / or with or for the purpose of non-default package options
3) unprivileged builds / installations, packages rooted anywhere in filesystem, completely self-contained dependencies
Having to use more than one package management system because one or the other doesn't do one of these things can be a real pain, and can make maintenance rather difficult.