Clojure microservices for JavaScript developers part 3

This series was co-written by Musa Barighzaai and Tyler Sullberg.

This is the third and final post in a series of posts for JavaScript developers about how to set up Clojure microservices. The previous posts were:

• Thinking in JavaScript vs Thinking in Clojure
• Creating Your First Clojure Microservice

Those previous posts are useful context, but you can clone the repo and jump into this post without reading them.

Using the Clojure test API

Compared to JavaScript, a convenient feature of Clojure is that it comes with a built-in unit testing library, clojure.test. In a Leinegen project by convention, you have an src directory which we know holds all the code, and for testing a test directory to separate your code from your tests. For any file under src that you want to test, you create a matching file under /test suffixing with \_test.

Imagine we have a file,adder.clj, in the following file structure of a Leiningen project.

src/
├─ adder/
│  ├─ adder.clj

To write unit tests for adder.clj we would create the following test directory:

src/
├─ adder/
│  ├─ adder.clj
test/
├─ adder/
│  ├─ adder_test.clj

Notice how the test directory mirrors our src directory, and how we add a \_test suffix on the file we want to test.

We want to test the following function in adder.clj.

(defn add-numbers [x y]
  (+ x y))

To write a test in adder.clj for the add-numbers function, you first bring in the Clojure Core testing framework.

(ns adder
  (:require [clojure.test :refer [deftest]))

Making use of the deftest macro, writing tests is as easy as:

(deftest test-add-numbers
  (is (= 4 (add-numbers 2 2))))

In our sample project, we have unit tests for all our HTTP endpoint handlers. Here’s an example of one (found in test/clojure_for_js_devs/handlers_test.clj) . This is testing the handler for our /counter path. The /counter path adds the requester’s IP address as a key to Redis and adds 1 to the value, keeping a count of the number of times an IP has pinged /counter.

(testing "counter-handler"
    (let [response "Counter: 44"
          req (->  (ring-mock/request :get "/counter"))]
      (bond/with-stub! [[redis/getKey (constantly 44)] [redis/incr (constantly nil)]]
        (is (= (handlers/counter-handler req {}) response)))))

There is a lot of going on here, so let us break it down for you.

First, we initialize a request object, req, for the endpoint we want to test (/counter). We are making use of a request mocking library that comes with ring-clojure (our HTTP server framework). This saves us time; we don’t need to write out a full request map.

Next, we use the library bond/with-stub!. In JavaScript the most popular testing framework is Jest. If you have worked with Jest before, bond gives you similar features. jest.mock() lets you mock modules and also state what you want the mock function’s return value to be. That’s what bond/with-stub does for us in Clojure. Since these are unit tests, we want to mock our call to Redis, specifically the keys getKey and incr. For getKey we want to return 44 anytime it’s called in this test, and similarly, for incr we want to return nil.

Finally, we make our assertion that our call to handlers/counter-handler will match our response, Counter: 44. Note that the final parameter in handlers/counter-handler is our Redis component, but in this test, we pass in an empty map {}. Since we are stubbing our Redis calls we can pass an empty map for this parameter, because Redis won’t be required in our test.

How about writing integration tests? In JavaScript, one of the useful features of Jest is the setup and teardown of tests. This feature comes in handy when writing integration tests. For example, if we had an application that queries a database of cities, you would do something like this in Jest:

beforeAll(() => {
  initializeCityDatabase();
});

afterAll(() => {
  clearCityDatabase();
});

test("city database has Vienna", () => {
  expect(isCity("Vienna")).toBeTruthy();
});

The beforeAll/afterAll allows you to run code before and after tests.

Clojure has built-in support for both test setup and teardown. It also makes use of fixtures. Here’s a full example:

(ns adder_test
  (:require [clojure.test :refer [testing use-fixtures]))

(defn my-fixture [f]
  ;; The function you want to run before all tests
  (initializeCityDatabase)

  (f)  ;;Then call the function we passed.
  ;; The function you want to run after all tests
  (clearCityDatabase)
 )
lang:clojure
(use-fixtures :once my-fixture)

(deftest city-db
  (is (= "'Vienna'" (IsCity))))

Here we are calling use-fixtures with :once, which means run my fixture only once, around all the tests in the namespace. You can also pass :each to run your fixture repeatedly for each test. use-fixtures here works the same as beforeAll/afterAll in Jest. We wrap our test (seen in my-fixture as (f)) with the method we want to be called before and after.

You can see a full example of this in the sample project, in test/clojure_for_js_devs/test_core.clj. In post 2 we discussed the purpose of a system-map. A system map allows our application to manage the lifecycle of each software component that it depends on. In our case, our components are our Redis connection and an HTTP server:

(defn- test-system
  []
  (component/system-map
   :redis (redis/new-redis "redis://localhost:6379")
   :http-server (component/using
                 (http/new-server "localhost" 0)
                 {:redis :redis})))

(defn- setup-system
  []
  (alter-var-root #'system (fn [_] (component/start (test-system)))))

(defn- tear-down-system
  []
  (alter-var-root #'system (fn [s] (when s (component/stop s)))))

(defn init-system
  [test-fn]
  (setup-system)
  (test-fn)
  (tear-down-system))

We have a method setup-system that uses the component library (from post 1) which starts the required components (HTTP server, and Redis). In addition to setup-system, we have tear-down-system, which runs component/stop to shut down all components once the tests have been completed.

Now, how do you run tests in Clojure? Without a test runner, you can call (run-all-tests) in your REPL to run all tests in all namespaces. Or, if you’re following along using Leiningen, you can call lein test.

You may have used test runners like Jest with features that run tests automatically when changes are detected in the code. The Clojure community also has various test runners with similar features. The most popular one is Kaocha. We’re using Kaocha in our sample project.

To set up Kaocha, start by adding it as a dependency in the project.clj file under your dev dependencies:

  :profiles {:uberjar {:aot :all}
                 :dev {:dependencies [[lambdaisland/kaocha "1.0.829"]
                                  [circleci/bond "0.5.0"]
                                  [ring/ring-mock]]}}

In JavaScript, you can create custom scripts for custom commands you want to run. For example, npm run prod:ci. Leinegen also has this feature known as an alias. We can create an alias test that loads the lambdaisland/kaocha dependency.

:aliases {"kaocha" ["run" "-m" "kaocha.runner"]}

Finally, add the Koacha configuration file. Create a tests.edn file in your root project directory with the following config:

#kaocha/v1
{:kaocha/color? true}

Now if you call lein test, kaocha will execute the tests and report the results.

Running Clojure tests in CircleCI

If you’re looking for an introduction to the importance of continuous integration (CI), I highly recommend checking out https://circleci.com/continuous-integration/. In this section we will go over our CI workflow and how to run Clojure tests in CircleCI,

In our sample project under .circleci/config.yml we have a workflow for testing our project every time we push a commit to our repo.

version: 2.1
jobs:
  build:
    docker:
      - image: circleci/clojure:lein-2.9.5
      - image: redis:4.0.2-alpine
        command: redis-server --port 6379
    working_directory: ~/repo
    environment:
      LEIN_ROOT: "true"
      JVM_OPTS: -Xmx3200m
    steps:
      - checkout
      - restore_cache:
          keys:
            - v1-dependencies-{{ checksum "project.clj" }}
            - v1-dependencies-
      - run: lein deps
      - save_cache:
          paths:
            - ~/.m2
          key: v1-dependencies-{{ checksum "project.clj" }}
      - run:
          name: install dockerize
          command: wget https://github.com/jwilder/dockerize/releases/download/$DOCKERIZE_VERSION/dockerize-linux-amd64-$DOCKERIZE_VERSION.tar.gz && sudo tar -C /usr/local/bin -xzvf dockerize-linux-amd64-$DOCKERIZE_VERSION.tar.gz && rm dockerize-linux-amd64-$DOCKERIZE_VERSION.tar.gz
          environment:
            DOCKERIZE_VERSION: v0.3.0
      - run:
          name: Wait for redis
          command: dockerize -wait tcp://localhost:6379 -timeout 1m
      - run: lein test

workflows:
  build:
    jobs:
      - build

The jobs key is where we define the jobs that can be used in our CI pipeline. Our pipeline has a single job, “build”, which will build our application and then run our tests. “Build” will use Docker images to set up our CI environment. Our Clojure application will run using a pre-built CircleCI Docker image: circleci/clojure:lein-2.9.5. These images are typically extensions of official Docker images and include tools especially useful for CI/CD. In our case, circleci/clojure:lein-2.9.5 comes with Clojure and lein installed. We also use the Redis Docker image since our integration tests will require hitting a Redis server.

Here is a review of all the steps for our “build” job:

1. We first check out our code from our repo.
2. We then wrap the lein deps call that fetches all our dependencies with caching. We don’t need to fresh install all our dependencies on every CI run, so caching will speed up our CI builds.
3. We then bring in Dockerize, a tool which gives our CI the ability to wait for other services to be available before running tests. In our case, we use Dockerize to wait for Redis before running our integration tests.
4. Finally, we run lein test to run all our tests.

We now have our CI pipeline running on every commit, automatically ensuring no one pushes a breaking change to our Clojure project.

And that’s it! You now have a Clojure microservice with basic tests using a continuous integration workflow. We hope you found this series valuable and have gained the confidence to use Clojure in your next project.