ClojureBridge Germany 2015

05.03.2015 Permalink

Programming languages breed cultures. They do it by means of their advocates who talk about them and differentiate them from other languages. They do it by how members of their communities interact with each other and with the people from the "outside". But I should be more precise: A programming language itself can't do anything. People create cultures, and using a programming language or not is just one type of boundary to make a distinction between "us" and "them".

Learning a programming language and its ecosystem sometimes requires quite a bit of frustration tolerance. The way members of a community interact with each other, the way how welcoming it is to people looking for guidance or people with different ideas makes a great deal of how attractive it is in the long term. But if you add fear of rejection or hostility to the hard work of learning (and sometimes failing), the community scares away even those who could potentially bring substantial contributions.

When I started to learn Clojure I recognized the openess within the community and a strong will to nurture a healthy mode of interaction among people. The other thing I immediatley enjoyed about the community is that its members have -- in summary -- the best ratio of good-vs-crap ideas in software engineering that I experienced in programming so far. In fact, those ideas changed fundamentally how I think about system design.

And then I came across ClojureBridge, a type of event that was new to me. Odds were high that I was missing an important idea, simply because its adoption in the Clojure community usually is a strong sign that it's valuable.

So I tried to understand the deeper purpose of ClojureBridge, I digged into diversity, understood how the lack of it was connected to how racism, sexism, ableism and all these other -isms work, and that such an event is one type of action to keep this insanity from growing further.

By the end of 2014, I played around with the idea of organizing the first ClojureBridge event in Germany, as I could rely on getting full support from my employer, itemis. I was very pleased to find out, that Silvia (innoQ) and Gerrit (codecentric) already had the same idea in their minds, so we formed the organizing team to distribute the work. Both got immediately full support from their companies, too.

However, one important question remained: will there be enough Clojure trainers available that help the attendees succeed in learning the language? And again, a very pleasing surprise: in only a few days we received more commitments than we have hoped for, especially from regional companies engaged in professional Clojure development like bevuta and doctronic, which -- in addition -- will help as sponsors.

So, we now have a date (June 12th to 13th), a building (thanks to codecentric), strong financial backup and the support of the local Clojure community. For all of this, I am very grateful, and I'm looking forward to the workshop in June!


Ideas for Clojure GUI libraries

15.12.2014 Permalink

Earlier this year I published a prototype to combine JavaFX with core.async to overcome some problems that I never solved in plain Java. Very recently, I sensed some interest in that prototype so I write a public post about my thoughts.

At the moment I have no plans to evolve async-ui as-is into a library, because it deals with numerous problems at once, i.e. it is not simple, and creating a GUI framework typically involves a huge amount of work. As it is today it just demonstrates that the idea of splitting UI code up into the following two conceptual functions

  1. build and update GUI from view state
  2. purely transform view state event by event
is an interesting approach for functional GUI programming, and that it can be combined with CSP style processes connected via channels to avoid nested callbacks, which - by the way - results in a crystal clear separation between GUI API access code and business related presentation logic.

It is the basis of a talk I give in January at ClojureD conf and it will remain a prototype. If, however, it turns out that many people actually want to have a more functional-style GUI programming experience in Clojure I would be willing to team up with others and put some effort in.

My ideas for such an endeavour are as follows:

1) I would rather focus on JavaFX than Swing, because JavaFX has a cleaner API, and Swing support has ended.

2) I would not like to create a "batteries-included" framework, because this would include numerous assumptions and trade-offs that might not fit actual needs and it's usually no fun to bend a framework into directions that its authors didn't anticipate in the first place.

3) One interesting aspect IMO is the transformation of Clojure data into the JavaFX scene graph. This is kind of the JavaFX version of React in browser DOM world. In async-ui this is currently done by the build multimethod that creates and configures controls, and by calling the setter functions to update data in controls. This transformation should be extensible in the sense that developers must be able to add transformation code for any individual control without touching existing namespaces (open-closed principle).

At the moment, with respect to this extensible transformation async-ui has two shortcomings: 1) the specification language in forml namespace is not extensible, 2) interactive development works, but rebuilds a Frame/Stage from scratch, so I would like to see a smoother way of merging changes into the scene graph.

It would take some exploration to identify the piece of non-trivial, reusable transformation related code that can be put into a library. At least such a library might provide some infrastructure and guidance. On top of this infrastructure, a benefit for real-world projects would be default transformations for widely used FX controls.

4) Application state management is a different aspect. Async-ui currently uses local state per Frame/Stage, not global state as in Om. I would like to keep the decision of how state is managed independent from the transformation problem. Cursors as in Om are IMO crucial for managing global state. A general cursor library would definitely be a useful asset. Sean Corfield started a ClojureScript cursor library for reagent which might act as a starting point.

5) Communication between GUI views, presentation logic and long-running code is another distinct aspect. My main requirements would be support for callback free code and a clear separation between GUI access code and presentation logic. I would like to keep the decision for the mechanism out of 3 and 4, but I'm not sure if this is possible.

In a concrete application I currently favor core.async over Rx-style libraries. I experimented with GUI and an Rx-like flatmap here. As you can see, Rx-style combinators help to generalize over different event sources and even treat the result of an asynchronously executed modal dialog just as another event source. But what I'm eventually after is a programming model that allows the programmer to stick to pure functions as much as possible. For this, I would try to get Rx or core.async to work more "in the background". With respect to GUI and asynchronity you get basically the same benefits from any of them, so I'd pick whatever is the more "mainstream" way in Clojure, which is core.async.

To sum it up, I think there a some useful pieces that one could build around Java GUI APIs to make them lend more towards functional programming. I hesitate to invest time here because I'm afraid that hardly anyone does Java GUI programming these days. If you think I'm wrong, drop me a mail or send me a tweet.


My quest for drastically simpler browser UIs

03.12.2014 Permalink

A web application built with significant amounts of JavaScript code running inside the browser is a viable UI technology choice. Even in a conservative environment, teams can today legitimately decide to ignore traditional Java web UI frameworks like JSF, Spring MVC, Wicket, Tapestry etc. in favor of purely JS based libraries like AngularJS, Ember or React which usually results in single-page applications (SPA)1.

TL;DR My ClojureScript prototype starts to show how nice UI code in the browser can look like: declarative, no callbacks, and an emphasis on purity.

The race

As I see it, teams coming from Java that turn to JavaScript for a large part of the application will face the following challenges:

JS looks similar to Java, but it is actually different in many ways. Teams working with e.g. Java in the backend and JS in the frontend need to have both skills available, which is why a backend JS engine like Google V8 with Node.js looks like an interesting option2 to reduce the technology stack.

JS as programming language is full of sharp edges and little flaws. ES6, once approved and widely available, will be an improvement, but it's far too early to state that JS will become a top-notch language for "programming in the large".

JS had until very recently3 no static type checker. Developers used to static type checking will have to change their habits to not shoot in their own foot. I don't think that a static type checker is a must to write well-crafted software, however it's a very useful tool to have in your belt when things get complex.

Let's face it: despite its problems, JS is the dominating web programming language and is here to stay. So it's quite understandable that many programming language communities are in the race to make their technology appealing to web UI developers. There are for example CoffeeScript, Dart, Elm, GWT, PureScript, Scala.js, TypeScript and god knows what, and yes: there is ClojureScript. All technologies have in common that they treat JavaScript as compilation target.

ClojureScripts architectural advantage

One particular challenge when building UIs is not JavaScripts fault, but a problem almost all languages without an explicit support for asynchronity share: callbacks in callbacks in callbacks, coordinated by shared mutable state. Many developers seldom face this problem in its entirety because we can often afford to avoid asynchronity outside of rich clients. But the browser is essentially a rich client programming environment with event handlers, a visual component tree a.k.a DOM and a single event-loop which must not be blocked. You want to react upon user input? Use a callback! You need a response from a remote call? Use a callback! You need to do some background calculation? Use a web worker and pass a callback! The problem here is that your logic is fragmented into several pieces, each wrapped by a callback. This makes understanding presentation logic harder. One callback is no issue, many will likely become a burden.

With core.async Clojure offers a practical implementation4 of an approach called Communicating Sequential Processes which can also be used in ClojureScript. There's a very nice introduction of core.async in the browser by Eric Normand. It allows you to eliminate callbacks almost completely, creating the illusion of a compact, sequentially executed process with local state. Combined with Om, a wrapper for Facebooks React library, ClojureScript promises to enable a drastically simpler way to encode UI functionality.

In the last few weeks I have built an architectural UI prototype, addressing some of the most recurring problems in enterprisey forms-over-data UIs like databinding, input validation or remote communication. My goal was to see how much work is actually needed to make "drastically simpler" happen in this scenario.

This work is on-going and can be followed on GitHub, I keep a demo running on my website. I'd like to point out three interesting results so far:

Concise specification

You can concisely specify the static looks of the UI, for example like so:

(panel "fields" :layout :two-columns
                :elements [(checkbox "private")
                           (textfield "name" :label "Full name")
                           (textfield "company")
                           (textfield "street")
                           (selectbox "city")
                           (datepicker "birthday")])	    

What you can see above are merely some nested function calls that eventually produce a data structure that is passed down to Om components. If you're able to assemble your UI out of simple components like those then such an approach is applicable and yields a textual specification of the UI.

No callbacks

You can get away without callbacks. Please take a look at the following piece of code which implements the reload action.

(defn <addressbook-reload
  [state event]
  (go (if (= :ok (<! (<ask "You will loose all your local changes. Are you sure?")))
        (let [{s :status addresses :body} (<! (http/get "/addresses"))]
          (if (= 200 s)
            (assoc-in state [:addresses :items] addresses)
            (Message. "Error loading addresses")))
        state)))
	    

Its intention is:

First, the user is asked for confirmation. If ok, a GET request is issued and, in case of success, the result is merged into the UI state.

You can test it in the Addressbook part of the running demo. For those unfamiliar with core.async I'll try to explain what's going on here:

The body of <addressbook-reload5 is wrapped in a go-block, which makes it a lightweight process that may pause when it accesses a channel. The immediate result of a go block is always an ad-hoc channel, that will eventually transmit the result of the body evaluation. The <! operation is a blocking read. A process reading from a channel is parked as long as there is no value available.

So the body inside <addressbook-reload first calls <ask which starts its own process and immediately returns a channel that will transmit the users response. The <ask process changes application state to show the question to the user. Then it reads from a channel that an event handler will put the users response on. Now, both processes are parked at <!: <ask is waiting for the user, <addressbook-reload is waiting for <ask. Once the response is available it is compared to the value :ok. If the user has confirmed to reload addresses from the backend the process issues a HTTP GET request, which again starts its own process, and does a blocking read with <!, and again the process <addressbook-reload is parked until a value is available. Finally, if the returned status is 200, the response contains addresses which are merged into the application state.

While all this might sound complicated at first, it allows us to write code that is as close to our intention as it can get. We just call ordinary functions and maintain full control of the flow of logic in one place.

Pure functions

If you scan the example code you can see that much of the behavior is expressed with pure functions. Functions without side-effects and any dependence on a global context are the simplest building blocks that we can have in an application, so we prefer to build as much as possible in this way. Presuming each piece of UI renders and maintains a piece of state, you can express presentation logic by event-wise application of pure functions of the form [state event -> state]. The pair Om+React is then the perfect tool to transform the updated state to a sequence of efficient DOM changes which we don't have to care about.

Despite the general preference for purity there are currently two exceptions to this:

Anyway, most of the logic like validation constraints, most actions, rules and the specification of the view can be expressed with pure functions that make automated testing of presentation logic a breeze.

Conclusion

It's clear that the style of UI programming as shown in this prototype is rigid, so I expect that a project team has to do its own exercise to come to abstractions that suit its needs. However, I'm confident that it is in general possible to reach such level of clarity for any type of frontend. I spent only about 10 working days on it, and with respect to what I wanted to reach I'm quite satisfied with the combination of ClojureScript, core.async and Om in the browser. ClojureScript has become a very appealing option that fulfills my hope for "drastically simpler" UIs.


1. I'm aware of the ROCA style as an alternative to SPAs, but unfortunately I did not find much practical advice in the blogosphere about it, except on an InnoQ GitHub account. Since the ideas sound interesting, hopefully someone can explain in better detail how it's done right.

2. Node.js is not without its own problems.

3. Facebook recently open-sourced Flow as a possible cure.

4. Go, the language, offers CSP too.

5. The '<' prefix in "<addressbook-reload" or "<ask" is a useful convention proposed by Eric Normand. It denotes that a function returns a channel, not a value.


Concise Om component composition

14.11.2014 Permalink

TL;DR React facilitates the creation of reusable components for single-page browser applications. To be useful in a wide variety of contexts components are often configurable. If you find yourself creating numerous composite components with much configuration data you can easily jump to a higher abstraction level which yields code looking like this:

(defn home-page
  [state]
  (page "home"
        :path nil
        :elements [(textbox "name" :label "Full name")
                   (checkbox "private" :label "Private?")]))

Forms over data

I have a very enterprise-y background. The style of UIs I encountered (and built) so far can be described by the term "forms-over-data": large number of forms composed out of a dozen of UI widget types, mostly static layout, visually a bit boring, but underneath full of complex validation logic and invariant checking. If you work in a user-centered development process1 for building the UI you'll usually see at some point in time sketches about how forms and their flow look like. You can get valid feedback with paper prototypes, but you learn more if users give feedback on the basis of software that kind-of already works. Building working prototypes is definitely more expensive than scribbling on paper so it should be the second step, but when you come to that task it would be great to be able to quickly translate the sketches 1:1 to code.

Om components

Let's switch over to Om2. Since it's based on React it promotes "componentization" of the UI, which means you'll likely create a toolbox of reusable components as you see recurring elements across UI sketches. Let's simulate this and build a form. I'll use a minimalistic example here, so I only introduce a configurable textbox and a checkbox:

(defn textbox
  [{:keys [value disabled message] :as state} owner {:keys [id label] :as opts}]
  (om/component
   (dom/div #js {:className "def-labeledwidget"}
            (let [input
                  (dom/input #js
                             {:id id
                              :className "def-field"
                              :type "text"
                              :value (or value "")
                              :disabled disabled
                              :ref id
                              :onChange #(om/update! state
                                                     :value
                                                     (-> % .-target .-value))})]
              (if label
                (dom/label nil
                           (dom/span #js {:className "def-label"} label)
                           input)
                input))
            (if message
              (dom/span #js {:className "error-message"} message)))))

  
(defn checkbox
  [{:keys [value disabled message] :as state} owner {:keys [id label] :as opts}]
  (om/component
   (dom/div #js {:className "def-labeledwidget"}
            (let [input
                  (dom/input #js
                             {:id id
                              :className "def-checkbox"
                              :type "checkbox"
                              :disabled disabled
                              :checked value
                              :onChange #(om/update! state
                                                     :value
                                                     (-> % .-target .-checked))})]
              (if label
                (dom/label nil
                           (dom/span #js {:className "def-label"} label)
                           input)
                input))
            (if message
              (dom/span #js {:className "error-message"} message)))))

Please note that the components receive static configuration data via the third function argument bound to opts. Apparently, I can refactor the component implementation to extract the label decoration by introducing a decorate function:

(defn- decorate
  ([label content]
     (decorate label nil content))
  ([label message content]
     (dom/div #js {:className "def-labeledwidget"}
              (if label
                (dom/label nil
                           (dom/span #js {:className "def-label"} label)
                           content)
                content)
              (if message (dom/span #js {:className "error-message"} message)))))


(defn textbox
  [{:keys [value disabled message] :as state} owner {:keys [id label] :as opts}]
  (om/component
   (decorate label message
     (dom/input #js {:id id
                     :className "def-field"
                     :type "text"
                     :value (or value "")
                     :disabled disabled
                     :ref id
                     :onChange #(om/update! state :value (-> % .-target .-value))}))))

  
(defn checkbox
  [{:keys [value disabled message] :as state} owner {:keys [id label] :as opts}]
  (om/component
   (decorate label message
     (dom/input #js {:id id
                     :className "def-checkbox"
                     :type "checkbox"
                     :disabled disabled
                     :checked value
                     :onChange #(om/update! state :value (-> % .-target .-checked))}))))

Nice, the rendering code as such can be made more modular by splitting it up into functions, no component ceremony needed for this. However, thanks to HTML these are still quite some lines of code for two little widgets3, but since the components are reusable I don't mind. Finally, here's a page component and the obligatory call to om/root:

(defn page
  [state owner]
  (om/component
   (dom/div nil
            (om/build textbox
                      (:name state)
                      {:opts {:id "name"
                              :label "Full name"}})
            (om/build checkbox
                      (:private state)
                      {:opts {:id "private"
                              :label "Private?"}}))))

(om/root
  page
  app-state
  {:target (. js/document (getElementById "app"))})

That's it, two configurable components and one composite page using them, everything is fine.

The rise of data

Let's assume now we have a sufficient number of components in our toolbox and someone asks us to create a prototype based on a pile of UI sketches. We might just proceed by creating more composite components in the style of our page, but let's first take a closer look to how it's implemented (please see above).

The code within page smells a bit like boilerplate. Essentially our page consists of components of different types, each with individual configuration. The configuration is passed as map via the :opts entry. The type is currently encoded as component function, passed as first parameter to om/build. If we instead encode the type in a keyword we can treat it as part of the configuration. This seemingly tiny step allows us to describe a page solely in terms of data.

Data has the nice property that it promotes concentration on what, independent of how or where it is used later on. Data can be designed in terms of the domain we want to express (here, the domain is the visual design of UI), allowing us to have a representation independent of technical API access. I call this data a specification4, or spec for short. Since React components already relate nicely to the bits we see in UI sketches the spec needs to consist only of custom configuration, the path into the state and the type of component.

There is one more benefit we can attain. Look at how the checkbox invocation is defined: the path keyword :private we use to create the cursor is just a keyword version of the id "private", which is redundant in this case. This congruence is quite likely, so it's reasonable to have a default path for navigating into the application state derived from the id5. If we create specs by invoking factory functions instead of just writing out the map literal we have a perfect place to encode these defaults.

Let's create some neat spec factory functions:

(defn textbox
  [id & {:keys [path label] :or {path [(keyword id)]
                                 label (clojure.string/capitalize id)}}]
  {::type  ::textbox
   :id     id
   :path   path
   :label  label})


(defn checkbox
  [id & {:keys [path label] :or {path [(keyword id)]
                                 label (clojure.string/capitalize id)}}]
  {::type  ::checkbox
   :id     id
   :path   path
   :label  label})


(defn page
  [id & {:keys [path elements] :or {path [(keyword id)]}}]
  {::type    ::page
   :elements elements
   :path     path})

There's really not much to these factory functions, perhaps the most interesting part is the :or in the map destructuring to derive the default values.

To be able to build components according to specs I introduce a component multimethod that expects a specification in the opts map and dispatches over the ::type value. This brings uniformity to the way components can be configured and built.

(defmulti component
  (fn [_ _ opts]
    (-> opts :spec ::type)))

(defn build
  [state {:keys [id path type] :as spec}]
  (om/build component
            (if path (get-in state path) state)
            {:react-key id :opts {:spec spec}}))

I have to slightly change the existing components to become methods that expect specs:

(defmethod component ::textbox
  [{:keys [value disabled message] :as state} owner {{:keys [id label]} :spec}]
  (om/component
   (decorate label message
             (dom/input #js {:id id
                             :className "def-field"
                             :type "text"
                             :value (or value "")
                             :disabled disabled
                             :ref id
                             :onChange #(om/update! state :value (-> % .-target .-value))}))))


(defmethod component ::checkbox
  [{:keys [value disabled message] :as state} owner {{:keys [id label]} :spec}]
  (om/component
   (decorate label message
             (dom/input #js {:id id
                             :className "def-checkbox"
                             :type "checkbox"
                             :disabled disabled
                             :checked value
                             :onChange #(om/update! state :value (-> % .-target .-checked))}))))


(defmethod component ::page
  [state owner {{:keys [elements]} :spec}]
  (om/component
   (apply (partial dom/div nil)
          (for [s elements]
            (build state s)))))

The result is that I can build any such component by evaluating (build state spec). Ok, done with the necessary changes. Now we're ready to make use of them:

(defn home-page
  [state]
  (page "home"
        :path nil
        :elements [(textbox "name" :label "Full name")
                   (checkbox "private" :label "Private?")]))


(om/root
  component
  app-state
  {:target (. js/document (getElementById "app"))
   :opts {:spec (home-page app-state)}})

Interesting! Look at the home-page function6. Without thinking too much about how to get rid of noisy HTML we just arrived at a more meaningful level of expressing the static aspects of a UI. There's a nice 1:1 relationship between a thing sketched on a piece of paper and a formal spec expression. I admit, as it is right now there's no way to specify layout information. But you can add this easily as CSS classname property for page or express it by having different types of container components.

The code resulting from transformation of numerous sketches to specs will now be quite readable and changeable. In fact, the spec factories form an internal DSL. I imagine one could even sit down with their user and live code, err... sorry, specify the UI interactively.

Conclusion

It's just a tiny idea and a trivial implementation that opens the door to make UI code more maintainable. But don't rush into it as this approach pays off only when you have some number of composite components that combine existing components. However, this is a typical characteristic for enterprise forms-over-data UIs, so I consider the idea to be valuable in this context.

I currently explore this and other ideas on a GitHub project named zackzack.


1. If you don't use steady user feedback to design your UI you run the risk of building the wrong application: one that doesn't fit user needs. This is the most expensive type of mistake you can make in your software project.

2. What I do here can likely be transferred to other React wrapping libraries like Reagent, Quiescent or Reacl.

3. I'm aware of the existence of libraries like om-tools, or kioo and sablono to ease this pain.

4. Instead of "specification" I could have used the term "model", but most people associate models with something graphical, so the latter term might cause confusion.

5. The underlying design principle is Convention over Configuration.

6. Defining the home-page spec within a function instead of just a def has the benefit to create state dependent specs, and it's also more friendly for REPL-based ClojureScript development.


Don't help the trolls

08.10.2014 Permalink

Here are my two cents about professionalized trolling on the internet.

I suspect some internet trolls recognized that there is an audience: trolls are servicing folks who are not only willing to accept harrassment and threat, but who enjoy to see the impacts of these unfair actions. For those, it's just a form of entertainment.

And the troll might think to himself: if there is an audience then there is a market. And something to earn. Mere attention, a feeling of power or perhaps even money. So, let's create a brand out of the little nasty me.

Thus, the troll seeks for victims to please the trolls miserable audience. Of course victims that are somehow well-known people. Otherwise, no one would care. And, in addition, the victim should belong to some group of which the troll can be safe that most people will stay silent when the violence happens against members of this group.

I think, the least thing that we non-trolls can do is to create a culture where the trolls can not be safe anymore that their actions have no unpleasant consequences for them.

I learned a lot by reading this NASCO document. I recommend it to any white man.

If you encounter some action in your environment that might worsen the situation for potentially discriminated people you have to speak up against it. If you keep silent it is implicit consent.

Which helps the trolls.