Red Echo

September 4, 2015

Things I’d still like to improve in this hypothetical kernel interface:

– access() and measure() are blatantly inefficient and really kind of terrible; you should just get that information for free when the message comes in, and if you want to inquire about object state, the call should let you ask about a whole batch of objects at once, to reduce the impact of syscall overhead.

– the mailbox design is sort of excessively clever, not likely to survive contact with the real world. I should just make different structs for incoming and outgoing messages.

– the idea of using a single syscall for all interactions with the outside world feels really nice, but I’m not sure I’ve gotten it right yet.

– I have a strong hunch that it will be important to resize queues some time.

– It feels wrong that there’s no way to cancel a message read and send some kind of fail signal back to the sender. Perhaps the solution would be to process send errors asynchronously, as messages received? But then you would need a bidirectional pipe, which I’ve been doing my best to avoid so far.

– extend() is the wrong name but I haven’t thought of the right one yet.

– every process can currently allocate memory willy-nilly, which feels like a contradiction with the overall exokernel style. Perhaps you should have to request a block of address space from a specific allocator… This would make an address space hierarchy easier, and would make it possible to provide feedback about memory pressure. right now it’s impossible to impose policy

– the previous draft, which I didn’t publish, had a notion I liked called a “bundle” – you could pack an array of objects up as a single object , send it around as an indivisible unit, and unpack it again later. It occurred to me that queues are not entirely dissimilar: what if you could create a pipe, push a bunch of stuff into it, then send the whole pipe with all of its contents to some other object? On receipt it would be a pipe with both send and receive permission.

– I still think there ought to be a way to share writable memory through some kind of transactional key-value mechanism.

– It makes me really happy that there is no file system.


I have no idea whether I’ll actually implement any of this, but I have three specific implementation concepts in mind providing constraints as I work on the design.

The first is naturally the idea of building out a full scale desktop/laptop computer operating system, suitable for all my daily computing activities – doesn’t every systems developer fantasize about throwing it all away and starting over? The capability / exokernel strategy has some significant security benefits, and the lack of a global filesystem, or any way to implement global mutations at all, means that every layer of the system can insulate itself against the layers underneath. It also provides a mechanism allowing the user’s shell to lie, cheat, and manipulate programs to make them do what the user wants, whether they like it or not, which makes me happy when I swear at stupid javascript crap.

Of course this will never happen. An embedded RTOS for microcontroller projects is small enough that I could feasibly implement it on my own, however, and I’ve actually done so in the past – in a limited, ad-hoc way – when I worked at Synapse.

This is the second project I think about as I consider the kernel architecture: a small, efficient kernel suitable for embedded realtime applications. There are several actions which can take advantage of an MMU’s virtual addressing features if present, but Trindle will get by just fine without it – while benefiting greatly from the kind of simple memory protection features found on high-end microcontrollers.

The third and simplest project would implement the Trindle kernel as a user-space library for Unix systems, which could help an application manage its parallel data processing needs by spawning a fleet of worker threads and managing their interactions. In this environment, there is no MMU, but we can still get basically what we need through judicious use of mmap/mprotect/munmap.

I don’t really know yet how useful this would be as an actual tool, but it seems like it would be easy enough to try it out and see what happened.