Packing and Shipping ClojureScript Applications

If you are building a web application for a business, inevitably you will be asked “can we make it into an app?” You suppress the urge to pedantically reply “but it already is an app.” Or, maybe you let frustration get the best of you and say it anyway. “No, no, like an app for your phone, not a website.”

Oh, you want an “app” you reply. Luckily you are already building your app with Clojure and ClojureScript so you have plenty of options for shipping your app in various packages. In this article we will look at the different ways to package and ship a ClojureScript application.

Before I begin, I feel obligated to say that this article is not about whether or not you should make a ClojureScript app. A lot of people will correctly point out that there is a mountain of JavaScript bloat that pollutes the web. Those people might also point out that a lot of JavaScript “apps” really don’t need to be apps at all. In a lot of cases user experiences are degraded by choosing JavaScript over a simple statically served page.

For the sake of argument lets assume that it is possible to create a good JavaScript app that improves the user’s experience. Hopefully after reading this article you will see that working with ClojureScript is delightful and it’s possible to build JavaScript applications without writing any JavaScript thanks to ClojureScript. It might be enough to change your mind about shipping JavaScript apps.

Getting Started

A great way to start a new project is to use the Luminus framework. In the spirit of frameworks like Common Lisp’s Caveman2 framework, Luminus is template generation tool that puts together a collection of best practices to help you get started in the right direction. You can easily generate new Luminus projects with Leiningen (install it now if you don’t have it).

Generate a new project
lein new Luminus shipping-cljs +kee-frame

This command will generate a new project called “shipping-cljs” as well as some boilerplate for setting up a single page application with kee-frame. Kee-frame builds on the re-frame library to offer URL directed state. That is to say the state of the app is determined by the data in the URL so users can share the state of their session with others by sharing a URL. Don’t worry if you are not familiar with these tools yet. Take a look at this commit to get an idea of what this command generates.

If we cd into this directory we can start up our Clojure and ClojureScript REPLs. Your development experience will be vastly improved by choosing to use a REPL. One advantage is that code reloading will be much faster. The other is that you can easily jump to the REPL to test out your ideas.

Start the Clojure REPL
lein repl

This will download all of the project’s dependencies the first time this command is run. After a moment a prompt is displayed that looks like user=>. Start the app by typing (start) and pressing enter.

user=> (start)

The server is started. If you visit localhost:3000 there will be a message that says ClojureScript has not been compiled and that you should run lein figwheel. Figwheel is a ClojureScript REPL. It offers live code reloading for a great development experience. Visiting localhost:3000 again now says “Congratulations, your Luminus site is ready!”.

Sometimes the figwheel prompot will hang. This is because figwheel is trying to establish a websocket connection with the browser. To resolve this clear the site data and refresh the page. The best solution is to view the page in private browsing mode and set up your browser to refresh all assets. This can be done in chrome by opening the developer console and refreshing. After this, the prompt should appear in the figwheel REPL.

Currently, Luminus uses lein-figwheel, however, the updated figwheel-main offers an improved experience. We will look at how to swap out lein-figwheel for figwheel-main later.

Figwheel is great for local development. However, if you look at the assets you are downloading in the developer tools menu, you will probably be appalled by the size. Out of the box this app weighs in at 10.3mb. That’s too large to ship to end users. Users on slow connections or mobile devices will suffer poor page load speeds. Luckily, ClojureScript offers a lot of optimization options at compile time. To keep code reloading lightweight during development, Figwheel does not optimize the compiled code much.

We can use cljsbuild for compiling our ClojureScript for production. Luminus includes an uberjar lein profile for compiling apps for production. First, let’s exit our REPLs. To exit the Clojure REPL, type (exit) and press enter. To exit the figwheel REPL, type :cljs/quit and press enter.

Compile project for deployment.
lein uberjar

This command will compile the Clojure and ClojureScript code. Everything will be bundled up into a Java JAR file. This Jar can be run with the Java runtime on your machine.

Run the compiled project
java -jar target/uberjar/shipping-cljs.jar

If we visit localhost:3000 again, we should see the same app that we saw before. This time if we inspect the asset sizes we are downloding we will see that they have shrunk considerably. After compiling a production build our assets are 1.2mb. Nearly one tenth the size. The bulk of the assets is a file called app.js weighing in at 993kb. That’s not a great size but it isn’t a terrible size either. Speaking from experience, some people will complain about a JavaScript file of this size.

Minification

One thing we can do to reduce the size of our app is to minify the asset. The first step is to add lein-asset-minifier to :plugins in project.clj. After that we need to add :minify-assets under the :uberjar profile in project.clj.

:minify-assets [[:js {:source ["target/cljsbuild/public/js/app.js"]
                      :target "target/cljsbuild/public/js/app.min.js"}]]

No we can add “minify-assets” to the uberjar prep-tasks.

:prep-tasks ["compile" ["cljsbuild" "once" "min"] "minify-assets"]

Take a look at this PR for a complete picture of the changes. If you looked at the diff in the pull request (you looked, right?) you will notice app.js is renamed app.min.js after it is minified. Take another look at the file diff in the pull request. There is a new file called home.html. Actually, this is just a copy of a file that already existed, but this new file is specific to serving the production build of the app.

Create a prod version of home.html
mkdir resources/html
cp resources/html/home.html env/prod/resources/html

Now the app can be built again via lein uberjar. With minification enabled we have realized a modest reduction in size from 993kb down to 984kb. Only 9kb. I suppose every bit helps. With advanced optimizations enabled ClojureScript builds are about as small as they can be in terms of size. The good news is that this tool can be used to minify other assets as well.

Code Splitting

This is where things start to get interesting. Browsers do a decent job of caching assets on their own. At the moment everything is going into a big blob of JavaScript called app.js. This file can be distilled into smaller pieces. For example, this file can be split into two files where one contains all of the libraries the application uses, and a separate file containing the implementation of these libraries that forms the application.

Why does this matter? If browsers are good at caching, browsers are smart enough to only request the parts of the application that have actually changed if a new release is published. ClojureScript allows you to split your code into multiple files at compile time. Here is what that looks like.

:modules {:app
          {:output-to "target/cljsbuild/public/js/app.js"
           :entries #{"shipping-cljs.app"}}}

What you don’t see here is that unless you instruct the compiler otherwise, it will also create a file called cljs_base.js and the module called :app will implicitly depend on cljs_base.js. This means the prod version of home.html also needs to be updated to include this new file and we need to minify the new asset.

Minify the new asset
:minify-assets [[:js {:source ["target/cljsbuild/public/js/cljs_base.js"]
                      :target "target/cljsbuild/public/js/cljs_base.min.js"}]
                [:js {:source ["target/cljsbuild/public/js/app.js"]
                      :target "target/cljsbuild/public/js/app.min.js"}]]
Add it to home.html
{% script "/js/cljs_base.min.js" %}

Now if we uberjar and run the app again we’ll see that the app is split between two files. This allows us to tell browsers to cache some files longer than others, improving page load times. This is useful if we plan to make changes that will affect the compiled output of app.js but not cljs_base.js. For example, if we don’t add new libraries cljs_base.js probably won’t need to change.

Take a look at this pull request to see the changes to project.clj to add code splitting.

Progressive Web App

Up until now we have focused on packing our ClojureScript app to ship it the traditional way in a web browser. Unfortunately, as the folks on the business side of things are quick to point out, it’s not an “app”. In other words, you can’t download it and install it on your phone or computer. In the back of your head you are thinking “but it is an app”. Of course it is, but now is not the time for pedantry. Save it for the next time someone wants to debate the use of the obviously mandatory oxford comma.

One way to solve this problem is to provide a progressive web app (PWA). The advantage of a PWA is that it is easy to convert an existing site into a PWA and it is easy to distribute. The disadvantage is that the scope of what a PWA can do is limited. For example, PWAs cannot access the filesystem. If your app cannot function offline a PWA is probably not a good choice. On the other hand if you are looking for a way to distribute something offline, like a game made with play-cljs, a PWA is a good choice for distribution.

Converting a ClojureScript app into a PWA can be accomplished with page-renderer. The page-renderer library automates creating the service worker required for chrome. The implementation of the PWA in this tutorial is inspired by https://github.com/Liverm0r/PWA-clojure which describes how to integrate Luminus projects with page-renderer.

One thing every app needs is an icon. Personally, I couldn’t make a logo to save my life, so I use https://favicon.io/ to generate icons. Put the icons in resources/public/icons.

Next add page-renderer to project.clj.

[page-renderer "0.4.6"]

Then create the file resources/public/manifest.json.

{
  "name": "Shipping CLJS",
  "short_name": "clj(s)",
  "scope": "/",
  "start_url": "/",
  "icons": [
    {
      "src": "/icons/android-chrome-192x192.png",
      "sizes": "192x192",
      "type": "image/png"
    },
    {
      "src": "/icons/android-chrome-512x512.png",
      "sizes": "512x512",
      "type": "image/png"
    }
  ],
  "theme_color": "#ffffff",
  "background_color": "#ffffff",
  "display": "standalone"
}

After this we need to tell the client to download the files for the PWA in <head>. Update resources/html/home.html.

<link rel="stylesheet" type="text/css" href="screen.css" media="all">
<link rel="manifest" href="/manifest.json">
<link rel="icon" type="image/png" href="/icons/favicon.ico"/>
<link rel="stylesheet" type="text/css" href="screen.css" media="all">

Take a look at this diff.

Next we need to register the service worker as seen in this diff.

(defn register-service-worker []
  (if (.-serviceWorker js/navigator)
    (-> (js/navigator.serviceWorker.register "/service-worker.js")
        (.then (rf/dispatch [:service-worker true]))
        (.catch (rf/dispatch [:service-worker false])))))

To generate the service worker with page-renderer we can add the following service worker handler to src/clj/shipping_cljs/routes/home.clj.

(defn service-worker [request]
  (pr/respond-service-worker
   {:script "/js/app.js"
    :sw-default-url "/"
    :sw-add-assets
    ["/css/screen.css"
     "/img/warning_clojure.png"
     "/icons/android-chrome-192x192.png"
     "/icons/android-chrome-512x512.png"]}))

Then we can add ["/service-worker.js" {:get service-worker}] to home-page. Refer to this diff.

At this point we have PWA. However, the visual cues to download the PWA can be very subdued in some environments like chrome on desktop and non-existent in others. Giving users special instructions to interact with their browser to make the PWA work is a tough sell.

We can use the beforeinstallprompt event to create a call to action that will only appear on browsers that emit this event. Since we are already using re-frame we should use it to tell our app when to display the call to action. The first step is reg-event-db and reg-sub for our install-prompt.

(rf/reg-event-db
 :install-prompt
 (fn [db [_ prompt]]
   (assoc db :install-prompt prompt)))

(rf/reg-sub
 :install-prompt
 (fn [db _]
   (:install-prompt db)))

Then we need a way to listen for the install prompt.

(defn register-install-prompt-listener []
  (js/window.addEventListener
   "beforeinstallprompt"
   (fn [e]
     (do (.preventDefault e)
         (rf/dispatch [:install-prompt e])))))

We’ll want to call this function in our init! function so the event listener is added when the page is loaded.

We also need a function to check if our app is running in standalone mode so we don’t display the download button after the app is installed on the user’s machine. The snippet below will covers any browser that is or is not safari.

(defn is-standalone? []
  (or (.-standalone js/navigator) (.-matches (js/matchMedia "(display-mode: standalone"))))

Now we can wire up the call to action to our navbar. We want to hide the install button after a user clicks on it so the button is not still displayed if the app is installed. We also want to put it back if the user cancels the install in case they change their mind. The snippet below is added to our navbar to accomplish this.

(when-let [prompt @(rf/subscribe [:install-prompt])]
 (if-not (or (is-standalone?) @installed?)
   [:div.navbar-item>button.button.is-primary
    {:on-click
     (fn []
       (do (.prompt prompt)
          (rf/dispatch [:installed true])
          (-> (.-userChoice prompt)
              (.then (fn [choice]
                       (if-not (= (.-outcome choice) "accepted")
                         (rf/dispatch [:installed false])))))))} "Download App"]))

Aside from giving the user a way to install the PWA, hiding and showing the button provides a subtle incentive to complete the download if the user is on the fence without being too annoying by taking away the big green button that otherwise feels a little out of place. I suppose you could package this up as a standalone function so you can stick it in all kinds of annoying places (looking at you Reddit), but I think your users will appreciate it if you don’t.

Take a look at this diff to see the button functionality in context.

The last the we need to do is hook everything up in our init! function.

First we need to explicitly set :installed to false.

(rf/dispatch-sync [:installed false])

Then register our service worker.

(register-service-worker)

Finally, register our install prompt listener.

(register-install-prompt-listener)

Each of these are used to determine whether or not the download button should be displayed.

The end result is a call to action button in our navbar that will only display if the uer’s browser supports installing progressive web apps. Additionally, the button will be hidden if the user has installed the app already. Look at this pull request to see all of the changes together.`

For a lot of potential apps out there a PWA should be more than enough. For example, I cannot think of a good reason Slack should be anything more than a PWA as the features available to PWAs cover all of Slack’s use-cases. In fact I always use the browser version of Slack and the only feature I cannot use if video chat although I am not aware of a technical reason video chat could not be enabled. It is even possible to package PWAs as apps that can be installed via the major apps stores for mobile devices.

What About Electron?

When I started writing this article my vision was to find a way to seamlessly progress from a ClojureScript app served from a web page to a progressive web app and finally to an electron app. These are all very possible independtly with ClojureScript (see this example), and it may be possible to seamlessly progress from one to the next with a different approach. Unfortunitely the approach I took for this article painted me into a corner where there was no obvious path from web app/PWA to electron without a lot of additional effort. As much as it pains me to admit it, I found myself burnt out on this little project by the time I got to this part and found myself drawn to other projects. As I write this, I am forcing myself to wrap up this article so I can clear it from my mind to focus on other things. I may come back to this and update it once I have encountered the proper motivation (inspiration or money). Instead I will share what did not work as well as some thoughts on how to accomplish the task. Maybe you can figure out how to make this work. If you do, be sure to contact me to let me know how you did it.

Problem #1 – Kee-Frame

I think Kee-Frame is really great from a philosophical standpoint of creating single page apps. The state of the app lives in the URL. It works flawlessly for creating standard web apps and single page apps. It also works just fine for an electron app that is served from a remote source. Unfortunitely, a fully featured app served from a remote source is kind of pointless as you still need to be connected to the internet. More importantly an app of this nature could be a vector for remote code execution. The electron docs explicitly recomend against running code from remote sources that will use native APIs. This prompts one to ask if electron apps should ever display content from a remote source as any remote source could be a bad actor. The best practice is to only execute code that is bundled with the app.

Loading the source code for the app from a file rather than a server is problematic for kee-frame because an electron app loading a local file will pass the fully qualified path to the file as the browser’s location.pathname. This value is used by kee-frame to manage the state of the app and it is not obvious how to tell kee-frame how to behave in this context. As a result, an exception is thrown due to an unmatched route and the app fails to load.

Fundementally, I appreciate what kee-frame stands for. It really makes sense for single page apps to manage their state via the URL because it allows users to share the state of their app with others by copying and pasting the URL. It’s a great idea because the lack of state-via-URL is a common critisism of single page applications. However, I could not figure out how to make kee-frame work correctly when location.pathname points to a local file rather than a server.

I don’t believe this is means that kee-frame cannot be used in the context of an electron app. However, the only solution I was able to come up with mean taking a completely different approach to the Luminus framework defaults by forgoing Clojure altogether and only using ClojureScriptto talk to native APIs to serve the app via localhost. This would work and be opaque to the user. It would also translate nicely between an app running on app running locally or on a server with the polyfills for APIs that can behave differently depending on the context of how the app is used. I like the idea of not needing to change how the client interacts with the host’s APIs because the host supplies the polyfill for how the APIs behave. This would require moving away from ring and selmar defaults in Luminus which requires quite a bit of rework to the Luminus build process. This also leads me to problem #2.

Problem #2 – Luminus Defaults

Using the Luminus framework I cannot see an obvious path to build an app that can host a web app, PWA, and electron app at the same time without significantly reworking the Luminus defaults or simply starting from scratch. Ring and Slemer are part of the Luminus defaults and are Clojure only libraries so we can’t run them with electron.

However, one option may be to package the Clojure code in a jar and have electron start the jar to serve the client. One could go as far as writing polyfills for route handlers that behave differently in client mode and server mode. If the app is backed by a database, PostgreSQL can be used in a server environment and h2 can be used in a client environment. The app could also offer client only endpoints for driving device hardware. I think some people will balk at this idea because it seems unnecisarly bloated. However, in a desktop only setting this may be a good option to minimize the number of differences between web clients and desktop clients.

An Alternative

I think something like the Macchiato framework may be a reasonable alternative for electron version of a ClojureScript app. This framework can be used to produce the electron version of the server side APIs our ClojureScript app consumes, or it could be used as a complete replacement to LUminous for a project that requires more deployment flexibility. The best of both worlds approach may be to produce a ClojureScript server that ships with our electron app while still using Clojure for our server side component. The downside to this is that we lose the ability to do multi-threading on the client and we also introduce some overhead by requiring the client to interact with the device via HTTP. We will also need to forgo html template rendering with Selmer.

I think this is an option worth pursuing, but for now I will leave it as an exercise to the reader. If you decide to pursue this and find success in doing so I am very interested to know about it. Contact me with a link to your example repo and I will add it here.

Conclusion

In conclusion, it is very easy to ship a traditional web app or progressive web app with the out of the box configuration provided by Luminus. Shipping an electron app requires more effort and possibly a different approach like using the Macchiato framework istead of or in addition to Luminus. Overall, ClojureScript is an excellent choice for resource constrained teams that do not have the time to maintain a native client app for multiple platforms. Hopefully this article sheds some light on how to approach building an app of this nature.