I had a startling thought on the bus this morning which led to a pretty exciting idea. What if I combined the work I’ve been doing on lightweight unikernels with all the work I did on Radian, turning Radian into a tool for building lightweight services?
I still think the Radian language is a good one; it’s clean and neat, and its semantics are constrained in just such a way that it’s easy to interact with asynchronous processes, and thus to split work up among multiple workers. As an experienced developer, I could do a lot of productive work with that language – but I think it’d be just as easy for a newbie to pick up as would python or javascript.
As a new language, though, it suffers from the unavoidable defect of being new: there is no language community, there are no libraries, there are no example programs, there are no tutorials, there is barely any documentation. Not if I made it my full-time job and worked the kind of hours I put in when I was doing Object Basic could I make up for that lack myself.
As a tool, it also has significant limitations. The constraints of its immutable dataflow semantics make it very difficult to interact with code written in other, less-constrained languages. That makes it very difficult to interact with the operating system and all of its libraries. Its parallel execution model and its dataflow semantics make it very difficult to debug using a conventional debugger: the traditional mode of stepping through line by line and watching the variables change is basically impossible, because there is nothing in the compiled code which cleanly matches up with the original source lines or variable names.
Finally, and worst of all, I was never able to find a problem for which Radian was a uniquely powerful solution. I looked into datamining, and then into numerics, but in each case I got bogged down in the immense amount of work necessary to get the core libraries up to the point that someone not already dedicated to the language would care to use it.
So, the new idea.
Unikernel-based services are still quite new, and *lightweight* unikernel services are still quite a cutting-edge activity. I’ve been having a great time poking around in the gritty guts of the x86 architecture but that’s because it reminds me of coding in the wild west of the late ’80s and early ’90s. Quite a barrier to entry!
What if I turned radian into a dedicated tool for building standalone microservices? The output would not be an executable program, but a bootable disk image you could start up inside a VM.
This would neatly turn all of radian’s disadvantages into strengths – or inevitable constraints of the target environment, at least. Can’t interact with system libraries? Well, isolation is the whole point of a lightweight unikernel service. Can’t step through with a debugger? Who cares, it’s probably running in the cloud anyway, so you debug it via log files – which is pretty much exactly the way I used to debug Radian programs anyway. Can’t link your favorite library into Radian? Doesn’t matter, just boot it up in a separate microservice, then make them sling packets at each other!
Radian’s strengths, meanwhile, would remain strengths. It’s all about slinging data between parallel asynchronous processes, and what else are microservices? Lightweight tasks are built right into the language, as is a synchronization mechanism which would fit perfectly on top of a socket-based communication mechanism.
Furthermore, there’s basically nothing else out there that does this. The only comparable system is MirageOS, which basically wraps OCaml and a bunch of unix libraries into a bootable VM. And that’s great, but it’s OCaml, and it’s a *much* heavier solution than I’d be presenting.
Radian, under this vision, would become shell scripting for the cloud: letting anyone with the tiniest trace of programming knowledge go from idea to running microservice in a matter of seconds. I am undoubtedly feeling unreasonably excited about this idea right now, but if I could accomplish this, it seems like it might be the most valuable thing I ever did with my career.
So. What would I have to do in order to bring this idea into reality? The radian compiler wouldn’t need any changes; all the work would be on the runtime side. I would certainly have to write some better documentation, though. I’d then replace the fleet kernel’s C runtime with the Radian runtime and rework the radian allocator and threads accordingly. I’d also have to write a little bit of code for emitting a disk image, and for invoking qemu on it.
As far as the radian standard library goes, I’d scrap the fancy but unfinished bignums package and bang out something simple and reliable based on good ol’ int64 and float64. The major piece of new work I’d have to do would be the development of a socket interface. Since I’d be going for an all-in-one runtime library, though, I could scrap the clunky ffi workarounds that used to make this sort of thing so laborious, and just build it right in! Call straight into the kernel, why not.
At that point… we’ve got sockets, tools for working with data going in and out of sockets, tools for managing complex asynchronous processes involving multiple sockets, and a tool that turns source code into a running VM in a snap of the fingers. What’s next? Time to play.
This would be an insanely formidable task if I were thinking about starting it from scratch, but I’ve already done all the hard parts! What I’ve just described sounds more like… assembling pieces.
Wow. Could I actually make all those years of effort pay off? I think I might try it.
