Janet is a modern Lisp with a rich standard library in a tiny package. Janet is quickly becoming my favorite Lisp for tasks I might have accomplished with bash/sh or python in the past thanks to its ability to produce small statically linked binaries. In this article we’ll explore remote procedure calls (RPC) with the spork package. Since RPC involves making calls over a network all code examples are divided by the client and server implementation in client.janet and server.janet. Follow along by running the examples on your own machine. You can view the example in once place as a gist.
Setup
The spork/prc package is an external library. Create a file called project.janet and add the following.
(declare-project :name "rpc"
:description "RPC test."
:dependencies ["https://github.com/janet-lang/spork.git"])
Afterwards, run the command jpm deps to download and install the spork package.
The Basics
Lets start by creating a basic RPC server and client to demonstrate how it works.
server.janet
rpc/server accepts a table of functions that a client can call. In this example we have function called :print that prints whatever the value of x is. Note that for ever RPC function we must also provide an argument for self, which is a reference to the object the :print method is implemented on. Yes, we are doing OOP in a Lisp.
(import spork/rpc)
(def functions
@{:print (fn [self x] (print x))})
(rpc/server functions "127.0.0.1" "9001")
client.janet
To create a client we need to tell it where the server is listening. We can also provide a name for our client. Since the client is an object with RPC methods that can be called the methods can be called like methods on any object created in Janet. For example if we bind the client to c we can call its :print method with (:print c "Hello world!). This will print “Hello world!” on the server.
(import spork/rpc)
(def c (rpc/client "127.0.0.1" "9001" "joe"))
# Will print on server
(:print c "Hello world!")
Returning Values
RPC is not much use if you can’t get data back. Before we explore how to go about doing that lets take a look at how we can make writing our RPC interface a little more ergonomic. Personally, I hate writing complex anonymous functions in the scope of a table like the previous examples. Luckily, we don’t have to. An alternative implementation where :print is defined outside of the functions table might look something like this.
server.janet
(defn print [self x]
(print x))
(def functions
@{:print print})
For the sake of brevity, I’ll assume you understand that any function defined in server.janet is added to the functions table like the example above.
So far we haven’t defined any functions capable of returning a value. :print calls print which writes to stdout and returns nil. Okay, I suppose you could say we haven’t written anything that doesn’t just return nil because nil is a value, just not a very interesting one.
server.janet
Lets add a function called add.
(defn add [self xs]
(sum xs)
Remember to add add to functions table under the :add key.
client.janet
We can call :add with our client to get the sum of the numbers we provide.
(:add c 1 3 5 7 9) # -> 25
This function will return the sum of the numbers, 25 in this case. If we want to, we can print the result.
(print (:add c 1 3 5 7 9))
Fibers over RPC
A fiber can be stopped and resumed later allowing a function to return multiple times. A fiber returns a value via yield. To make a fiber yield a value one must resume it.
server.janet
Lets make a new RPC function called fiber that returns a new fiber. In that fiber is a function that yields values in a loop. Each time we resume it we get the next value.
(defn fiber [self]
(fiber/new (fn []
(for i 0 5
(yield i)))))
Remember to add this to the functions table under :fiber.
client.janet
Here our client calls the :fiber method and binds it to fiber. Afterwards it resumes the fiber in a loop printing each value as it resumes.
(let [fiber (:fiber c)]
(for i 0 5
(print (resume fiber))))
What if we need to send values to a remote fiber while also reading from it? For reasons I can’t begin to comprehend I spent my Saturday indulging this thought exercise of enabling bi-directional (bidi) data flow over RPC fibers in Janet. This is what I came up with.
server.janet
This RPC function accepts a remote fiber from the client that we’ll call rfiber. It resumes the remote fiber in a loop and then increments and yields the value back to the client.
(defn bidi [self rfiber]
(fiber/new (fn []
(while true
(yield (inc (resume rfiber)))))))
Observant readers might point out that this loop is infinite, which is true, but remember that the fiber is suspended after yielding so it isn’t really an infinite loop unless it is also resumed infinitely.
client.janet
Here we define the fiber on the client which will act as the remote fiber that passes values to the server. To avoid the infinite loop issue we’ll resume the fiber in a try. On each iteration we’ll push the values to an array. When the fiber is dead it will raise an error. At that time we can print the values and break from the loop.
(def f (fiber/new (fn []
(for i 0 5
(yield i)))))
(let [bidi (:bidi c f)
arr @[]]
(while true
(try # we can't see if the fiber f is dead because the rpc server is consuming it not the client.
(let [v (resume bidi)]
(if v # v is nil if fiber is dead and we don't want that.
(array/push arr v)))
([err] (do
(pp arr)
(break))))))
Sending a File
Up until now we focused on things that are debatably interesting but dubiously useful. Wouldn’t it be nice to be useful? Transferring a file is certainly useful.
server.janet
Something not so obvious is that not all of Janet’s core library can be serialized for use over RPC. The file API for example cannot be serialized for RPC. Intuitively, I thought wrapping the file API in a fiber for the client to consume was a convenient way to transfer a file over RPC but it doesn’t work.
If you just wanted to send the whole file at once you could do that without involving fibers. However, this will not work in situations where the file you want to transfer is particularly large or perhaps continuously appended like a log. To enable file streaming over RPC I expose an interface similar to the file API where the client first “opens” the file which stores the file handle in a table on the server called handles. You could think of this as a table of virtual file handles that can be used to look up the actual file handle when the client wants to read from the file.
Each time the client reads the file it also supplies the virtual handle so the server knows which file to read from. This allows us to transfer a file line by line or bit by bit instead of all at once. The virtual handle is created by the function make-handle which generates a random string that can be used as the key in the handles table to store the reference to the actual file handle on the server.
(def handles @{})
(defn make-handle []
(string/join (map | (string/format "%02x" $) (os/cryptorand 12))))
(defn open [self file & mode]
(let [handle (make-handle)]
(put handles handle (file/open file))
handle))
(defn read [self handle what]
(file/read (handles handle) what))
client.janet
Lets start with a simple example where we simply open and read the entire file.
(let [file (:file/open c "./foo.txt")]
(pp (:file/read c file :all)))
Here we “open” the file on the remote host and receive the virtual file handle back. The virtual handle is bound to file and then we call the :file/read method, supplying the virtual handle to the server so it knows which file to read. The interface is very similar to the actual Janet file API.
Here is a more complicated example where we stream a remote file called foo.txt and write it to local file called bar.txt line by line.
(let [file (:file/open c "./foo.txt")
to (file/open "./bar.txt" :a)]
(while true
(let [line (:file/read c file :line)]
(if line
(file/write to line)
(break))))
(file/close to)
(print (file/read (file/open "./bar.txt") :all)))
Now that we are done, we should close the client connection by calling (:close c) on our client.
Conclusion
Although spork/rpc is simple it opens the door to great deal of possibilities for creating distributed systems with Janet. Additionally the interfaces created can be nearly identical to a given API as I have demonstrated with the file transfer example. The complete examples of the client and server are available as gist.