In September 2013 we blogged about why we’re supporting Typed Clojure, and you should too! Now, 2 years later, our engineering team has made a collective decision to stop using Typed Clojure (specifically the core.typed library). As part of this decision, we wanted to write a blog-post about our experience using core.typed.

The reason that we decided to stop using core.typed was because we found that the cost of using it was greater than the benefit we gained. This is a subjective view, of course, so we will detail our reasoning below.

The core.typed library is part of the Typed Clojure project. It is a library that adds optional typing to Clojure code. Core.typed allows the developer to add type-annotations to Clojure forms, and then a type-checking process can be run to verify the type-information of your program.

First of all, some background on our project. The repo for our main app contains 93,983 lines of Clojure code. The src directory of our main app contains 369 namespaces, of which 84 have been annotated with core.typed, and 285 that have not.

Our codebase is therefore loosely divided into 3 groups: code that is type-annotated, code that has type annotations with the ^:no-check modifier, and code that does not have type annotations.

The problems that we have hit with using Typed Clojure are slower iteration time when programming, core.typed not supporting the entire Clojure programming language, and using third-party libraries that have not been annotated with core.typed annotations.

Slower Iteration Time

When developing in Clojure, our team writes code interactively at the REPL. We like to be able to write code, and then immediately run the tests. When using core.typed on our codebase, running the type checker takes a long time. On my machine, (a 3GHz i7 MacBook Pro) type-checking a single namespace takes 2 minutes. This leads to a situation where the code gets ahead of the type annotations. I typically write code and corresponding tests at the same time, iterating quickly. When using core.typed I would then add type annotations to the code that I had written, and then run the type checker, since the two minute runtime was too slow to run interactively.

The reason that the type checker takes 2 minutes is because it re-scans all of the files in our project to collect the type annotations. We did change our core.typed setup for a period to only re-scan the current file when running the type checker. This improved the run time for the type checker to be near instantaneous, but this prevented us from working on two interdependent files. We found that modifying the type annotations in one file and running the type checker on the other file was not possible with this setup - the changes to the first file were being ignored.

Core.typed Does Not Implement The Entire Clojure Language

My personal grievance when using core.typed is that it does not implement the whole Clojure language. Some functions cannot be type checked, for example get-in, comp, and some uses of or. This means that after writing some code, we would often have to go back and re-write some forms to appease the type checker.

One particular example of this was where I needed to pull out a form that took a function f and returned:

(comp keyword f name)

(composing my function f between calls to name and keyword). This code cannot be checked by core.typed, so instead I was had to extract the form out into a top level function and add a type annotation (and add a comment to explain why I was doing it):

(t/ann keyword-transformer
  [(t/IFn [String -> String]) -> (t/IFn [t/Keyword -> t/Keyword])])
(defn- keyword-transformer [f]
  (fn [x] (-> x name f keyword)))   

My frustrations are not because there is code that core.typed cannot check, that’s understandable. The problem is that we cannot easily differentiate situations where our code has type errors from situations where core.typed cannot check the code. The error messages produced are the same in both cases.

Here is an example of the error that is produced from the comp example above:

Polymorphic function comp could not be applied to arguments:
Polymorphic Variables: x y b
Domains: [x -> y] [b ... b -> x]
  (t/IFn [(t/U t/Symbol java.lang.String t/Keyword) -> t/Keyword]
         [java.lang.String java.lang.String -> t/Keyword])
         [java.lang.String -> java.lang.String]
         [(t/U clojure.lang.Named java.lang.String) -> java.lang.String]
Ranges: [b ... b -> y]
with expected type: [t/Keyword -> t/Keyword]    

When faced with problems like this, it is not clear to the user if the code being checked contains a bug that causes the type check to fail, or if the code has exposed a limitation in what core.typed can check. My first attempt to work around issues like this would be to re-write the code in a few different forms to see if I could make the type checker pass. This is where the 2 minute iteration time becomes frustrating - I would make a small change to factor out a form and add type annotations, then run the type checker. If the type checker failed again, I would have to repeat the process of extracting forms and adding type annotations. Each iteration would impose a further 2 minute wait time.

After a few failed iterations I would assume that the problem was a limitation of core.typed itself, and add a ^:no-check annotation to my code, and move on. I found that all too often I was choosing to add a ^:no-check annotation rather than taking the time to diagnose the underlying issue. Every form with a ^:no-check annotation weakens the guarantees of type system, and allows bugs to bypass the type checker.

Another problem that we hit at CircleCI is where a function has a type signature that cannot be type-checked with core.typed, so we had to annotate the return type as Any. Our apply-map function if a good example of this – the function is like apply but it can be used for functions that expect a set of keyword arguments as the rest parameters:

(t/ann ^:no-check apply-map
  [(t/IFn [t/Any -> t/Any]) t/Any * -> t/Any])

(defn apply-map
  "Takes a fn and any number of arguments. Applies the arguments like
  apply, except that the last argument is converted into keyword
  pairs, for functions that keyword arguments.
  (apply foo :a :b {:c 1 :d 2}) => (foo :a :b :c 1 :d 2)"
  [f & args*]
  (let [normal-args (butlast args*)
        m (last args*)]
    (when m
      (assert (map? m) "last argument must be a map"))
      (apply f (concat normal-args (apply concat (seq m))))))   

We have not been able to get this function to type-check correctly with core.typed, so we have had to annotate as returning Any (the type-signature for apply is quite complicated). Using the Any type like this is infectious - all type information is lost when there is a call to apply-map - so any functions that call apply-map cannot type-check correctly. We invariably have to annotate those functions with the ^:no-check` flag`, which means that any functions that callapply-mapare not type-checked. At Circle we use keyword arguments in this style a lot: there are 115 calls toapply-map`` in our code-base, which would be 115 functions that we cannot type check, which again weakens the whole type system.

Writing type annotations for some functions can be very difficult – when searching through our code I found one comment in particular that I felt was representative of the feelings we have when having to add a ^:no-check annotation:

(t/ann ^:no-check all-granted-by
  [t/Hierarchy t/Keyword -> (t/Coll t/Any)])
;; sigh. i still can't figure out how to write moderately 
;; complicated core.typed checks for stuff like this.

Third Party Code

None of the third-party libraries that we use are annotated with core.typed. This means that we had to create and maintain a list of type annotations for these third party libraries ourselves. We maintain a file that contains the type-annotations for the third-party code that we depend on. Any time we add a call to a library function that we had not called before, we need to write an annotation for the function. The type-annotations that we write are not checked by core.typed (they are marked ^:no-check). So we need to be very sure that that these annotations are correct. If these annotations were added to the libraries themselves, then they would be type checked. We have offered to add our annotations back to some projects, but for project maintainers who don’t use core.typed themselves there is little to be gained from accepting a patch to add type annotations. The additional maintenance overhead is not worth it, and core.typed would add an additional dependency to the library.

The Future For Us - schema.core

For now, we have disabled our unit-tests that would run the type checker over the namespaces that we have annotated with core.typed. We are loath to remove to type annotations from our codebase, even though they are not checked, because of the valuable documentation that they add to the code.

The code that we find most value in type checking is field access in maps, since maps are “stringly typed” (or more accurately for Clojure, “keywordly typed”). In these areas we have been adopting use of Prismatic Schema in the manner described by Jessica Kerr in a recent episode of the Cognicast. We add schema.core definitions to the functions that we want to type-check. These are mostly functions that take a map as one of the arguments. When running our tests we enable runtime schema checks using the the schema.test/validate-schemas test fixture. For some functions we also enable the ^:always-validate flag to ensure that the schema is checked in production also.

Using schema.core allows us to add schema checks to our code gradually, without some of the problems that we encountered with core.typed. For this flexibility we have to forgo compile-time type checking and instead rely on runtime type checking, which delays the point at which we find type errors in our code. The schema definitions do give us the same structured documentation that core.typed gives us, which is something that we really like.