Care to hear about our lord and saviour Meson?

Both of your quoted problems are mutually incompatible: dependency management isn't the job of the compiler, it's a job for the build or host system. If you want to keep writing makefiles, be prepared to write your own `wget` and `git` invocations to download subprojects.

Meanwhile, Meson solves the dependency management problem in a way that makes both developers and system integrators/distributions happy. It forces you to make a project that doesn't have broken inter-file or header dependency chains and cleans up all the clutter and cruft of a makefile written for any non-trivial project, while making it trivial to integrate other meson projects into your build, let other people integrate your project into theirs, and provides all of the toggles and environment variables distribution developers need to package your library properly. You can really have your cake and eat it too.

https://mesonbuild.com/

The contractors have a huge productivity advantage out of the gate - modern hardware, any software, no mandatory VPN that blocks genuinely useful websites (MDN!).

I'd juxtapose these two sentences.

As I understand things. Those who know Nix swear by it. You can declare a development environment which will provide the toolchain and the libraries you need to build your software.

Some things do seem inelegant about Docker containers. e.g. Building the images with Dockerfiles feels fragile. Running containers means high friction to accessing the build environment from the host machine.

Those downsides aside, AFAIU the VSCode devcontainers aim to provide that "wow it just works" experience that the Nix people love, without having to pay the steep learning curve of learning Nix.

I thought this too for a long time, but the more I'm exposed to languages with "proper" dependency management the more I appreciate the C way of just copying external library sources into the project (and I only consider libraries which make this easy, e.g. if they come with their own complex build system files they already lost - just give me a bunch of headers and source files, and maybe a readme with a list of configuration defines).

And why is that? I find it really cumbersome.

Those who tried it and swear by it.

Those who tried it and swear at it along with words like 'never again'.

Those who tried it, and avoid it in favor of Guix, but would use Nix if they had to.

nix on the other hand, I loved the concept and idea, but it was just too much of a 'stop sign' followed by 'we don't do that kind of user activity here' to be usable. I tried for a few weeks to bend our wills together, but the systems' will won and I walked away.

I suspect my experience with guix would have been remarkably similar if I hadn't been put off by the speed - I love scheme, but guile has always seemed the second slowest implementation ever, and I suspected that was the cause for it taking so long to do anything in guix.

This is only usually a problem if you want to download something large or do a large update though. If you're just downloading a small program or you update your system frequently, it's quite reasonable to use.

As far as the "stop sign", I have never really run into that with Guix, and I did run into that with Nix. The fact that Scheme - unlike the Nix language - is not purely functional, I think encourages users to do what they want, even if it goes against the spirit of the functional package manager.

Please don't take this as "you're wrong" or to somehow invalidate your experience. In the two years of running Guix as my main system, the only time I've thought something was slow was in building Chromium from source (which took all night to compile). Everything else I never noticed. Certainly not downloads. Those were fine for me both in the US and in Europe. Your experience contrasts so sharply with mine, I wonder what the difference is?

Sometimes downloads are fine though. I think it's an issue of load on the servers that will cause downloads to be slow sometimes.

For several decades, I have never compiled any C project otherwise than with auto-generated dependencies between files, which are generated by the compiler and used by make.

I never write by hand dependencies for a makefile. (I also never write by hand the list of source files, but I let make search for them and choose the appropriate compiler)

It is true that many open-source projects have horrible makefiles, but I have never understood why anyone has written such makefiles, when the GNU make manual explains the right way and that explanation already existed at least a quarter of a century ago.

I.e., you can point a .devcontainer at a Dockerfile that you've already got working for your environment and specify standard configurations for environment variables, vscode extensions, and other bootstrap configurations so that the startup friction for a new developer working on the code is reduced to the minimum and the manual steps they need to compete to get started are documented and standardised in the devcontainer format, but you don't lose anything by doing so.

First environmental category: your dev environment itself.

What platform are you using? What architecture are you using? Do you need a VM? How will you mount filesystems? How will you handle networking - through the VM or on the native host? How will you handle HTTPS proxies - on the docker client, on the docker server, in the VM? Do you need a custom DNS resolver? Do you need additional applications for your platform, like AWS or GCP CLI tools and credentials? Do you need more applications, some of which may have specific version requirements (for example, Homebrew famously does not support old versions of software)?

Second environmental category: CI/CD.

How should I do a build, test, and deploy from my laptop? How should I do a build, test, and deploy from a CI/CD server? How can I make sure I get the same result from either? How can I reduce the number of dependencies in the deployed form, without getting different results at runtime? How can I re-use things from my local machine in the CI/CD so that I don't have to maintain two different ways of doing the same thing?

- In the best case: you have no problems at all, and everything works automatically.

- In the worst case: one of a hundred different things can break your dev environment. You end up spending an inordinate amount of time just "maintaining your dev environment", rather than writing application code. And the more developers there are, the higher the probability of time is spent on "environment work" rather than "making the product".

- Besides all of that dev environment work, you end up replicating your whole process in a completely different CI/CD system, and now you're maintaining two separate yet identical things.

A solution to all of these environmental factors combined is Devcontainers: https://containers.dev/overview Because it's one standard, all sorts of tools can now take advantage of it. This means different devs can use different tools to solve the same problem with the same config file.

It's obviously not infinitely scaleable because you could end up with name conflicts for object files, or some modules might require custom build settings. But it works well enough to nest my own projects 3-4 layers deep without issue.

(1) There's no recursive option; each dependency has to be pulled/pushed/etc. manually.

(2) Each manual pull/push/etc. of a subtree has to be explicitly path-specified and remote-specified.

There's just no easy way to `git subtree pull --prefix=specificdependencysubdir https://github.com/whatever/gadgets main --squash` for each of your dependencies, and each of their dependencies, etc. It's a tedious manual process for everything.

Best case scenario, you make a .PHONY target of `updatesubtrees` and manually keep it up to date with the directory structure and remote URLs of each dependency.

Why don't they scale? I used them for some years for a team of ten or so people who all first had to learn to understand how they worked so might that be the problem?

There's not one giant issue with submodules. There are dozens of small annoyances and footguns. I have yet to see a problem that is better solved by git submodules that can't be solved another way.

Of course Git kinda sucks and isn't good at hosting toolchains. So I'm putting a heavy emphasis on "should".

I thought devcontainers were merely a way of telling VSCode to host your dev environment in a Docker container. I'm confused about what distinction you are drawing here.

Right, which is gross. It gets worse when you start talking about using them practically in an enterprise-ish environment. There, they end up being a less effective Xen-style programming interface. It's too bloated for most cases. The distinction I make is building with docker (for cross compilation or whatever) vs hosting your entire dev environment in a container.

I don't really see the difference between just building via docker, and doing static analysis, incremental builds, running tests, etc inside docker. Surely the goal in all these scenarios is the same: a reproducible environment for every developer on the project?

It's hard to take stuff out of a running container; and it's hard to access files/programs on the host from within the container.

There may be advantages to running inside a container; but it largely feels like when there are easier ways for quickly making programs available to the host (e.g. virtual environments, or asdf version manager), then most tools aim for that first.

Bind mounts let you easily move files in/out of the container (and are already set up by devcontainers). And the whole point is to _not_ access programs on the host, you want that isolation so that the environment is reproducible and everything you need to build is defined in the dev container.

It just needs your build toolchain and libs.. you don't need to use the shell from the container to run random unix utils or curl for instance.

Bloated how? Which cases?

Any C app (or even Python app, since python libs like to depend on C libraries) with non-trivial dependencies get very annoying to configure across a range of distros (even worse if you include MacOS and/or Windows).

`sudo apt install libpng-dev` vs `sudo dnf libpng-devel` etc.

Rather than document and test all those different configs, devcontainers is a really easy way to avoid this pain for example applications or ones that will only ever ship to one distro/OS. And if you're running on Linux atleast, there's literally no overhead (containers are just processes tagged with C-groups, after all).

Doing it inside a container solves the portability problem, but you're still using a Linux distribution's package manager to get the dependencies of your project, which makes no sense to me at a fundamental level, even if it "works" in practice.

Is vendoring the only somewhat sane way of doing dependency management in C?

I would hope that tools like conan and vcpkg solve this now on the developer side? I don't have much experience with them though.

You still have to deal with libc though, which means you likely need to containerize to avoid issues with old versions or distros that use something other than glibc (musl or bionic, for example).

It's a lot more complex with C/C++ to build fully static binaries than it is in something like rust or GoLang.

cgroups require code to implement. This is overhead. "Literally" is not the word you should be using. Perhaps "little" overhead or "minimal" overhead?

IMO I'm using it correctly here though, let me explain.

Overhead is originally a business term that refers to an ongoing cost. Yes there is a small amount of code in the kernel that has to run, but my naive understanding is that this code also runs for processes that are not in a container (the kernel still needs to check whether the process is in a namespace or not). Additionally, I've never seen a benchmark that shows a workload performing worse when cgroups are applied. I'm happy to be proven wrong here but if this is the case, then there is no ongoing cost (and thus no overhead).

* granted I did just spend half a day last week figuring out that WSL environment variables are not correctly applied to the containerEnv, but otherwise they've been solid

The tool chains for ARM GCC on windows are a real pain to path correctly for multiple systems. Not to mention the same issue for OpenOCD, Debug, etc.

Bazel solves this really nicely, I know some people have strong opinions on it but I cannot recommend it enough

https://bazel.build/

Docker vs. devcontainers: You got my point. It isn't about trying to force people to use devcontainers and vscode, it's about maintaining and sharing a development environment. You could just open a shell in this container and swap the base image out for whatever you need (also pointed out in the article). I myself also don't use vscode devcontainers but just exec into the running container and, e.g., use docker compose (or podman or whatever works best) :)

EDIT: The reason for showcasing devcontainers is that you get an IDE with code completion, format on save and all the other goodies "for free", whereas any plain docker or nix setup requires that you do this on your own. In my career, I've seen way too many people editing code in notepad or notepad++, making tons of mistakes that can be avoided by whatever IDE. I'm not saying vscode is best, I'm saying right now it is the easiest IDE to set up.

It is recommended at work now, where I also can use "the old way" which is a series of Makefiles. I've never had a problem with the Makefiles, but they've been hand-tuned for a few decades, and errors would be reported immediately by the build server. They check if you have the right compilers and then just work. Contrast this with Docker, where you always have the right compilers and they only work sometimes.

While using Docker, if even one source file changes while the compilation is happening, the VM will simply hang and refuse to compile or accept any break keystroke. Outside of Docker, a file change sometimes works without error and sometimes you'll cause a minor problem that a new iteration of "make <whatever>" fixes in a few seconds. Worst case? "make clean", "make <whatever>", no reboot required.

After force quitting the Docker-based build, I try "machine stop" and "machine start" but often encounter errors and need to reboot my machine and/or reinstall Docker, such as this gem from last week, "No connection could be made because the target machine actively refused it."

I don't know if Docker has an equivalent to "make extraclean" or "make distclean" or "sudo cut the bullshit && let me use you again", so I'm already uncomfortable restarting a build after rebooting/reinstalling. (That you need to manually delete container directories after uninstalling is also troubling.) I know some data are stored in the container in a mixed-persistence way that uses the existing filesystem. It's confusing to me why some generated output (.pdb, .exe, .dll) are accessible outside the container while others (.o) are evidently not used if you decide later to rebuild without Docker. Thus, the difference when switching is often recompiling 5 files in a few seconds vs 500+ in a few minutes, so there's a short-term incentive to try to keep using Docker, where I then waste time trying to solve Docker instead of just recompiling with the old method. Any semblance of "Flow" I have after getting a Docker error gets annihilated.

If our local container server goes down (it's not often but has happened), I am unable to build because some of our dependencies aren't FOSS/widely available.

I've had issues related to services, prerequisites, permissions, mounts, certificates, massive network login delays... When I hear the constant praise for containers, it gives me pangs of imposter syndrome, because my experience has been awful and mentally taxing. I find I'm constantly fixing tools instead of using them.

I can't necessarily blame Docker, but twice in the past 3 years I've needed to install a fresh OS within a week after installing it because of the problems above.

Docker's main selling point? It fixes a problem we don't really have. Every developer at my company has essentially the same computer---an overpowered Ryzen/Quadro combo in a black tower with a small collection of preinstalled essentials like MSVC bintools, VCS software, network drives preconfigured---unless they specifically request another for build purposes, i.e. Mac or Sun. In the latter case, that developer (or small group of developers) is in charge of builds on that host OS. In the one place it matters, we have one version of a compiler expected in a specific directory, and this is well-documented and changes once every few years.

It's possible I completely misunderstand Docker or have a misconfigured system (whatever THAT means, I thought the whole point was to eliminate problems caused by local customizations), but for anyone to tout it as fool-proof would mean they have severely underestimated the technical (in)abilities of fools.

Docker, to me, often feels like bringing in and using a CNC machine when all I usually need is a sharpening stone and chisel.

-----

Also, I would be quite uncomfortable using GitHub Actions to compile C for a microcontroller. There's simply too long of a delay between changing a config YAML and getting an error, fixing it, waiting for the next error. Plus, despite the "low overhead" of Docker, if you do this long enough on GitHub (especially with a non-Linux host OS) you will run into server fees.

And after you set up a GitHub Action, are you writing in VS Code, waiting for the remote computer to compile, downloading the artifact, syncing that with VS Code so you can debug, flashing to the target with your own set of (locally installed) tools, and finally debugging?

Contrast that with a local install... Edit. Save. "make", "make flash", gdb. In 20% of the time, with no server costs.