thatmattbone.com

One of the interesting things about processes in Unix is that they are tree structured. This is easily seen with the pstree command; init is the root of all processes. Several things, can, however go wrong with this tree. Defunct or zombie processes show up when a child process dies and the parent is unaware. Likewise, orphan processes occur when the parent dies and the child is still running. Despite the names, neither is a tragedy. Zombie processes are removed from the process table when the parent dies, and orphans become the children of init. Thus, the strict tree structure is maintained.

Compare this to a multithreaded system. Threads are not forked from other threads. They have no explicit parent other than the enclosing process or address space. If we treat threads like objects in a garbage collected OO environment (i.e. we need to have a reference to a thread in order for it to survive), we quickly see they are not tree structured like Unix processes. One thread can create five new threads, pass the references to other threads, and disappear. There is no parent/child relationship, and the newly created threads needn’t care about the status of the thread in which they were originally created. The representation of these relationships is not a tree, but rather a graph that evolves over time.

What does this have to do with erlang-in-lisp? The light-weight processes in Erlang are much closer to the graph-nature of multithreaded systems, and with our current approach, we will have to map this graph onto a the Unix process tree.

The simplest approach is to have each process wait for its children to complete before exiting or returning. This should probably be our first approach. However, in larger or long running systems, this could become a burden, with many short-lived processes hanging around and taking up memory (though not cycles) while their longer-lived children run to completion.

As I suspected, make-socket-pair worked just fine. I discovered this yesterday when I set the input and output buffer sizes to 1. Then, I had the sinking sensation that while I’d already tried flushing the buffer, I had been flushing the wrong side.

I think one of the side items that should come out of erlang-in-lisp is a simple iolib tutorial.

Well the first week of SoC has come and gone. I’m disappointed with my progress. I had hoped to have some simple ping/pong examples working by the end of the first week, but I’ve yet to resolve some issues with iolib and message passing. Realistically, I think I can fix these problems and have the examples working today or tomorrow.

My problems stem from local domain sockets in iolib. I can express what I’d like to do in C with socketpair/fork/read/write (and get it working), and the iolib API should, of course, make this much simpler. But for whatever reason, the streams created by make-socket-pair, don’t seem to be connected; when I write to the one side, I cannot read from the other. Late last night I started to suspect this was some buffering issue, and fired up the slime inspector on these objects. I didn’t learn much. I also looked closely at the iolib code, and everything seems to be in order. The problem is on my end. My next step is to try using them as binary streams (I’ve been treating them as character streams) ala the tcp server in philip-jose. Either that or raise hell in #lisp.

I view these hiccups as noise rather than signal, and I still think we’re on track for an interesting summer. I have a much more fundamental question about our approach in the next posting.

Though the summer of code does not officially start until tomorrow, I’m pleased to report that we have made some initial progress.

To start, we have common-lisp.net site with a git repository. The site has instructions on how to check out the code. Our (mostly empty) erlang-in-lisp project works with asdf and is easily loaded with slime-load-system. Some dependencies are kept in version control, but most are checked out with the makefile in the deps directory. This makefile is pretty repetitive, and I’m thinking of replacing it with something a little more maintainable (perhaps an scsh script).

I’ve spent some time thinking about our intial fork-based concurrency approach. Though there are portable libraries for some posix operations like file and directory management, I’ve been somewhat surprised that there isn’t a Common Lisp implementation for all of the posix functionality (though it seems there have been some thoughts in this direction). Still we should be able to rely on the implmentation specific posix packages or roll our own with cffi for now.

Also, there is a little test-misc.lisp file serving as a catch all for miscellaneous tidbits. Right now there are some tests from cl-muproc in there that I think we should get working with our fork framework and IPC messaging by the end of the week.

I still have many questions, though, and I’ll be posting a list of them soon.

I’ll be participating in the Google Summer of Code working on the erlang-in-lisp project. I’ll be tracking my progress on this blog (so things will get noisier). I had originally created a separate blog, but Google decided it was a spam blog, and I don’t really see much point for this kind of separation at the moment.