The problem goes deeper. I can't remember who coined the term, but all "implerative" (imperative declarative) languages share the same issue. I don't care if it's JSON, XML, TOML, or YAML, we shouldn't be interpreting markup/data languages. GitHub actions are a good example of everything wrong with implerative languages.
Use a real programming language, you can always read in JSON/YAML/whatever as configuration. Google zx is a good example of this done right, as is Pulumi.
Kris Nóva said it best: "All config drifts towards Turing completion."
Oh man, i have a similar issue with NixLang. Though i know it's not "implarative". Many days i just want to write Nix in my preferred language. I wish Nix had made a simple JSON based IO for configuration, because then i could see what the output of something is - and generate the input state from some other language.
Really frustrating. Nix works.. but i just don't see the value, personally. And this is after living on NixOS for ~3 years now, with 4 active Nix deploys in my house.. i just don't like the language.
I'm currently building this (plus more) - the happy path of what you're talking about is almost complete. There are fundamental issues preventing what you're talking about being used as a complete replacement for NixLang: you'd need every possible language installed/available on the builder machine in order to build packages, and lazy evaluation would completely break (merely evaluating all of nixpkgs takes hours). So you do ultimately need a primary language. That being said, for devops-like stuff there is no reason to have that limitation.
I wanted to use Nickel, but it turns out that it can't do everything you'd need it to do to completely replace NixLang. So right now I'm bikeshedding on what to use instead (and desperately trying not to invent something), in other words it's definitely being renamed. Either way there's a bash script in the `test` dir that shows the general concept.
I see Nix as a powerful way to write config files. It is purely functional, so the only thing it does is create a build recipe. That build recipe is then run by other Nix tooling.
A .nix file is either a config file itself or a function that returns a config file or a function. By passing in enough parameters, you get the configuration. I've not seen as clean a way of doing this anywhere else. Guix uses Guile which is a full programming language and can probably have side effects. They use something called G-Expressions which is not quite clear to me.
The problem is (to me) it's entirely obtuse. I can't call the function and get back some configuration - which is insane to me. You have to pass in all sorts of state, and you have a lot of difficulty producing the exact same state as your config in question would see in a real execution. Or at least i do. I even asked on several forums and the answer kept boiling down to "Well, it's just not easy. Sometimes not possible." What's the point of it being functional?
Ie yea, i can load up the Eval and call my config func - but what about the params? Well now i have to generate the params. Some of them might be easy, but some are difficult as hell - and if they differ now executing my func in the Eval is not producing the same output (or failing entirely) as it does when i run it "for real".
Nix in practice felt like all of the problems of imperative languages but wrapped in a nice functional wrapper. It was functional without any of the real benefits of functional - to me.
Eg i can't easily get the same input and pass it into a function to produce the same output. To be able to view a function as a simple slice of functionality that i can inspect, debug, etc. They have get access to the entire universe (nixpkgs/etc), a huge stdlib, etc - and you need to recreate all of that if you want to use the function.
The parameters you pass in define your dependencies. For a program to compile it needs the compiler and that is a complicated dependency. One might think that only passing the paths to the dependencies would be enough. That way the inputs could be much simpler indeed. I guess there's room for a simpler Nix.
While i will instantly switch to Nickel for the type system once Nickel is available, i do think Nix could get a lot further by just having better tooling.
Notably error reporting is atrocious, but an interactive debugger would be amazing too. Ie to set a breakpoint and hop into an eval at your breakpoint. Would help immensely.
Still i just can't get behind a dynamic typing for anything remotely complex.. which i would describe Nix as. I have been counting down the days for Nickel.. it's been a long wait.
Nix can read JSON, there's a deserializer as one of the builtins you can call. So you can make a bridge where Nix reads your JSON and does something with it, and you can generate the JSON externally like you want. It's how things like poetry2nix work.
Completely opposite experience for me. I think Nixlang is exceptionally well designed and makes sense for the usecases it wants to cover, and it is exactly what I would expect from a DSL tackling the problems it tackles.
"Implerative" - thank you for this, this is the term I've been searching for to describe the weird blending of the two things.. I immediately Googled it and saw that it has previous uses as well, I would love to know who originated the concept. I see so many times, confusion and arguing about what is imperative and declarative, to the point where I question the value of the terms any longer.
FWIW, I have flirted with my own DSL implementations in a few cases. Certainly, language design is much more complex, but I also felt that once you understand enough of EBNF/parser generators (and some of the simpler alternatives), this is a very powerful option as well.
I'm also pretty against DSLs, although they do rarely have uses cases. For an example of why DSLs can be bad, look at Dockerfiles contrasted with Buildah. The former makes tons of assumptions, especially when to perform layer checkpoints. The latter is just a script in Bash or whatever your language of choice.
For the curious, this might be it:
"I've cracked our marketing code, y'all!
Pulumi: Implerative Appfrastructure" [1]
@funcOfJoe, Joe Duffy: CEO of Pulumi
I've always wondered why we seem to have implemented a whole programming language in yaml or json for so many CI/CD systems rather than just writing quick python scripts to describe the logic of a particular build step, then MAYBE using a JSON or XML file to enumerate the build steps and their order, like:
Sure, that's orchestration, though. The problem with GHA is the sheer amount of expressive power that it has. If you need to do dynamic stuff then that should be in a "pre-workflow" step, written however/in whatever you please, that emits the actual workflow.
Why shouldn't the python script be the discrete workflow step? It could be mounted on some file system which has checked out the git at a particular commit with a particular tag, then runs whatever tasks are required to validate or deploy the project
For tools that allow configuration in either JSON or Javascript (like eslint), I prefer the JS version. The syntax is similar but has much more flexibility, like being able to use environment variables or add comments.
Pulumi was also a good tool when I was doing kubernetes deployments.
Use a real programming language, you can always read in JSON/YAML/whatever as configuration. Google zx is a good example of this done right, as is Pulumi.
Kris Nóva said it best: "All config drifts towards Turing completion."