In the post Overloading lambda, I gave a translation from a typed lambda calculus into the vocabulary of cartesian closed categories (CCCs). This simple translation leads to unnecessarily complex expressions. For instance, the simple lambda term, “λ ds → (λ (a,b) → (b,a)) ds”, translated to a rather complicated CCC term:

apply ∘ (curry (apply ∘ (apply ∘ (const (,) △ (id ∘ exr) ∘ exr) △ (id ∘ exl) ∘ exr)) △ id)

(Recall from the previous post that (∘) binds more tightly than (△) and (▽).)

However, we can do much better, translating to

exr △ exl

which says to pair the right and left halves of the argument pair, i.e., swap.

This post applies some equational properties to greatly simplify/optimize the result of translation to CCC form, including example above. First I’ll show the equational reasoning and then how it’s automated in the lambda-ccc library.

Equational reasoning on CCC terms

First, use the identity/composition laws:

f ∘ id ≡ f
id ∘ g ≡ g

Our example is now slightly simpler:

apply ∘ (curry (apply ∘ (apply ∘ (const (,) △ exr ∘ exr) △ exl ∘ exr)) △ id)

Next, consider the subterm apply ∘ (const (,) △ exr ∘ exr):

  apply ∘ (const (,) △ exr ∘ exr)
≡ {- definition of (∘)  -}
  λ x → apply ((const (,) △ exr ∘ exr) x)
≡ {- definition of (△) -}
  λ x → apply (const (,) x, (exr ∘ exr) x)
≡ {- definition of apply -}
  λ x → const (,) x ((exr ∘ exr) x)
≡ {- definition of const -}
  λ x → (,) ((exr ∘ exr) x)
≡ {- η-reduce -}
  (,) ∘ (exr ∘ exr)

We didn’t use any properties of (,) or of (exr ∘ exr), so let’s generalize:

  apply ∘ (const g △ f)
≡ λ x → apply ((const g △ f) x)
≡ λ x → apply (const g x, f x)
≡ λ x → const g x (f x)
≡ λ x → g (f x)
≡ g ∘ f

(Note that I’ve cheated here by appealing to the function interpretations of apply and const. Question: Is there a purely algebraic proof, using only the CCC laws?)

With this equivalence, our example simplifies further:

apply ∘ (curry (apply ∘ ((,) ∘ exr ∘ exr △ exl ∘ exr)) △ id)

Next, lets focus on apply ∘ ((,) ∘ exr ∘ exr △ exl ∘ exr). Generalize to apply ∘ (h ∘ f △ g) and fiddle about:

  apply ∘ (h ∘ f △ g)
≡ λ x → apply (h (f x), g x)
≡ λ x → h (f x) (g x)
≡ λ x → uncurry h (f x, g x)
≡ uncurry h ∘ (λ x → (f x, g x))
≡ uncurry h ∘ (f △ g)

Apply to our example:

apply ∘ (curry (uncurry (,) ∘ (exr ∘ exr △ exl ∘ exr)) △ id)

We can simplify uncurry (,) as follows:

  uncurry (,)
≡ λ (x,y) → uncurry (,) (x,y)
≡ λ (x,y) → (,) x y
≡ λ (x,y) → (x,y)
≡ id

Together with the left identity law, our example now becomes

apply ∘ (curry (exr ∘ exr △ exl ∘ exr) △ id)

Next use the law that relates (∘) and (△):

f ∘ r △ g ∘ r ≡ (f △ g) ∘ r

In our example, exr ∘ exr △ exl ∘ exr becomes (exr △ exl) ∘ exr, so we have

apply ∘ (curry ((exr △ exl) ∘ exr) △ id)

Let’s now look at how apply, (△), and curry interact:

  apply ∘ (curry h △ g)
≡ λ p → apply ((curry h △ g) p)
≡ λ p → apply (curry h p, g p)
≡ λ p → curry h p (g p)
≡ λ p → h (p, g p)
≡ h ∘ (id △ g)

We can add more variety for other uses:

  apply ∘ (curry h ∘ f △ g)
≡ λ p → apply ((curry h ∘ f △ g) p)
≡ λ p → apply (curry h (f p), g p)
≡ λ p → curry h (f p) (g p)
≡ λ p → h (f p, g p)
≡ h ∘ (f △ g)

With this rule (even in its more specialized form),

apply ∘ (curry ((exr △ exl) ∘ exr) △ id)

becomes

(exr △ exl) ∘ exr ∘ (id △ id)

Next use the universal property of (△), which is that it is the unique solution of the following two equations (universally quantified over f and g):

exl ∘ (f △ g) ≡ f
exr ∘ (f △ g) ≡ g

(See Calculating Functional Programs, Section 1.3.6.)

Applying the second rule to exr ∘ (id △ id) gives id, so our swap example becomes

exr △ exl

Automation

By using a collection of equational properties, we’ve greatly simplified our CCC example. These properties and more are used in LambdaCCC.CCC to simplify CCC terms during construction. As a general technique, whenever building terms, rather than applying the GADT constructors directly, we’ll use so-called “smart constructors” with built-in optimizations. I’ll show a few smart constructor definitions here. See the LambdaCCC.CCC source code for others.

As a first simple example, consider the identity laws for composition:

f ∘ id ≡ f
id ∘ g ≡ g

Since the top-level operator on the LHSs (left-hand sides) is (∘), we can easily implement these laws in a “smart constructor” for (∘), which handles special cases and uses the plain (dumb) constructor if no simplifications apply:

infixr 9 @∘
(@∘) ∷ (b ↣ c) → (a ↣ b) → (a ↣ c)
⋯ -- simplifications go here
g @∘ f  = g :∘ f

where ↣ is the GADT that represents biCCC terms, as shown in Overloading lambda.

The identity laws are easy to implement:

f @∘ Id = f
Id @∘ g = g

Next, the apply/const law derived above:

apply ∘ (const g △ f) ≡ g ∘ f

This rule translates fairly easily:

Apply @∘ (Const g :△ f) = prim g @∘ f

where prim is a smart constructor for Prim.

There are some details worth noting:

• The LHS uses only dumb constructors and variables except for the smart constructor being defined (here (@∘)).
• Besides variables bound on the LHS, the RHS uses only smart constructors, so that the constructed combinations are optimized as well. For instance, f might be Id here.

Despite these details, this definition is inadequate in many cases. Consider the following example:

apply ∘ ((const u △ v) ∘ w)

Syntactically, the LHS of our rule does not match this term, because the two compositions are associated to the right instead of the left. Semantically, the rules does match, since composition is associative. In order to apply this rule, we can first left-associate and then apply the rule.

We could associate all compositions to the left during construction, in which case this rule will apply purely via syntactic matching. However, there will be other rewrites that require right-association in order to apply. Instead, for rules like this one, let’s explicitly left-decompose.

Suppose we have a smart constructor composeApply g that constructs an optimized version of apply ∘ g. This equivalence implies the following type:

composeApply ∷ (z ↣ (a ⇨ b) × a) → (z ↣ b)

Thus

  apply ∘ (g ∘ f)
≡ (apply ∘ g) ∘ f
≡ composeApply g ∘ f

Now we can define a general rule for composing apply:

Apply @∘ (decompL → g :∘ f) = composeApply g @∘ f

The function decompL (defined below) does a left-decomposition and is conveniently used here in a view pattern. It decomposes a given term into g ∘ f, where g is as small as possible, but not Id. Where decompL finds such a decomposition, it yields a term with a top-level (:∘) constructor, and composeApply is used. Otherwise, the clause fails.

The implementation of decompL:

decompL ∷ (a ↣ c) → (a ↣ c)
decompL Id                        = Id
decompL ((decompL → h :∘ g) :∘ f) = h :∘ (g @∘ f)
decompL comp@(_ :∘ _)             = comp
decompL f                         = f :∘ Id

There’s also decompR for right-factoring, similarly defined.

Note that I broke my rule of using only smart constructors on RHSs, since I specifically want to generate a (:∘) term.

With this re-association trick in place, we can now look at compose/apply rules.

The equivalence

apply ∘ (const g △ f) ≡ g ∘ f

becomes

composeApply (Const p :△ f) = prim p @∘ f

Likewise, the equivalence

apply ∘ (h ∘ f △ g) ≡ uncurry h ∘ (f △ g)

becomes

composeApply (h :∘ f :△ g) = uncurryE h @∘ (f △ g)

where (△) is the smart constructor for (:△), and uncurryE is a smart constructor for Uncurry:

uncurryE ∷ (a ↣ (b ⇨ c)) → (a × b ↣ c)
uncurryE (Curry f)    = f
uncurryE (Prim PairP) = Id
uncurryE h            = Uncurry h

Two more (∘)/apply properties:

  apply ∘ (curry (g ∘ exr) △ f)
≡ λ x → curry (g ∘ exr) x (f x)
≡ λ x → (g ∘ exr) (x, f x)
≡ λ x → g (f x)
≡ g ∘ f

  apply ∘ first f
≡ λ p → apply (first f p)
≡ λ (a,b) → apply (first f (a,b))
≡ λ (a,b) → apply (f a, b)
≡ λ (a,b) → f a b
≡ uncurry f

The first combinator is not represented directly in our (↣) data type, but rather is defined via simpler parts in LambdaCCC.CCC:

first ∷ (a ↣ c) → (a × b ↣ c × b)
first f = f × Id

(×) ∷ (a ↣ c) → (b ↣ d) → (a × b ↣ c × d)
f × g = f @∘ Exl △ g @∘ Exr

Implementations of these two properties:

composeApply (Curry (decompR → g :∘ Exr) :△ f) = g @∘ f

composeApply (f :∘ Exl :△ Exr) = uncurryE f

These properties arose while examining CCC terms produced by translation from lambda terms. See the LambdaCCC.CCC for more optimizations. I expect that others will arise with more experience.

As Haskell is a higher-order functional language in the heritage of Church’s (typed) lambda calculus, it also supports “lambda abstraction”.

Sadly, however, these two forms of abstraction don’t go together. When we use the vocabulary of lambda abstraction (“λ x → ⋯”) and application (“u v”), our expressions can only be interpreted as one type (constructor), namely functions. (Note that I am not talking about parametric polymorphism, which is available with both lambda abstraction and type-class-style overloading.) Is it possible to overload lambda and application using type classes, or perhaps in the same spirit? The answer is yes, and there are some wonderful benefits of doing so. I’ll explain the how in this post and hint at the why, to be elaborated in futures posts.

Generalizing functions

First, let’s look at a related question. Instead of generalized interpretation of the particular vocabulary of lambda abstraction and application, let’s look at re-expressing functions via an alternative vocabulary that can be generalized more readily. If you are into math or have been using Haskell for a while, you may already know where I’m going: the mathematical notion of a category (and the embodiment in the Category and Arrow type classes).

Much has been written about categories, both in the setting of math and of Haskell, so I’ll give only the most cursory summary here.

Recall that every function has two associated sets (or types, CPOs, etc) often referred to as the function’s “domain” and “range”. (As explained elsewhere, the term “range” can be misleading.) Moreover, there are two general building blocks (among others) for functions, namely the identity function and composition of compatibly typed functions, satisfying the following properties:

• left identity:  id ∘ f ≡ f
• right identity:  f ∘ id ≡ f
• associativity:  h ∘ (g ∘ f) ≡ (h ∘ g) ∘ f

Now we can separate these properties from the other specifics of functions. A category is something that has these properties but needn’t be function-like in other ways. Each category has objects (e.g., sets) and morphisms/arrows (e.g., functions), and two building blocks id and (∘) on compatible morphisms. Rather than “domain” and “range”, we usually use the terms (a) “domain” and “codomain” or (b) “source” and “target”.

Examples of categories include sets & functions (as we’ve seen), restricted sets & functions (e.g., vector spaces & linear transformations), preorders, and any monoid (as a one-object category).

The notion of category is very general and correspondingly weak. By imposing so few constraints, it embraces a wide range mathematical notions (including many appearing in programming) but gives correspondingly little leverage with which to define and prove more specific ideas and theorems. Thus we’ll often want additional structure, including products, coproducts (with products distributing over coproducts) and a notion of “exponential”, which is an object that represents a morphism. For the familiar terrain of set/types and functions, products correspond to pairing, coproducts to sums (and choice), and exponentials to functions as things/values. (In programming, we often refer to exponentials as the types of “first class functions”. Some languages have them, and some don’t.) These aspects—together with associated laws—are called “cartesian”, “cocartesian”, and “closed”, respectively. Altogether, we have “bicartesian closed categories”, more succinctly called “biCCCs” (or “CCCs”, without the cocartesian requirement).

The cartesian vocabulary consists a product operation on objects, a × b, plus three morphism building blocks:

• exl ∷ a × b ↝ a
• exr ∷ a × b ↝ b
• f △ g ∷ a ↝ b × c where f ∷ a ↝ b and g ∷ a ↝ c

I’m using “↝” to refer to morphisms.

We’ll also want the dual notion of coproducts, a + b, with building blocks and laws exactly dual to products:

• inl ∷ a ↝ a + b
• inr ∷ b ↝ a + b
• f ▽ g ∷ a + b ↝ c where f ∷ a ↝ c and g ∷ b ↝ c

You may have noticed that (a) exl and exr generalize fst and snd, (b) inl and inr generalize Left and Right, and (c) (△) and (▽) come from Control.Arrow, where they’re called “(&&&)” and “(|||)”. I took the names above from Calculating Functional Programs, where (△) and (▽) are also called “fork” and “join”.

For product and coproduct laws, see Calculating Functional Programs (pp 155–156) or Functional Programming with Bananas, Lenses, Envelopes and Barbed Wire (p 9).

The closed vocabulary consists of an exponential operation on objects, a ⇨ b (often written “$b^a$”), plus three morphism building blocks:

• uncurry h ∷ a × b ↝ c where h ∷ a ↝ (b ⇨ c)
• curry f ∷ a ↝ (b ⇨ c) where f ∷ a × b ↝ c
• apply ∷ (a ⇨ b) × a ↝ b (sometimes called “eval”)

Again, there are laws associated with exl, exr, (△), inl, inr, (▽), and with curry, uncurry, and apply.

In reading the signatures above, the operators ×, +, and ⇨ all bind more tightly than ↝, and (∘) binds more tightly than (△) and (▽).

Keep in mind the distinction between morphisms (“↝”) and exponentials (“⇨”). The latter is a sort of data/object representation of the former.

Where are we going?

I suggested that the vocabulary of the lambda calculus—namely lambda abstraction and application—can be generalized beyond functions. Then I showed something else, which is that an alternative vocabulary (biCCC) that applies to functions can be overloaded beyond functions. Instead of overloading the lambda calculus notation, we could simply use the alternative algebraic notation of biCCCs. Unfortunately, doing so leads to rather ugly results. The lambda calculus is a much more human-friendly notation than the algebraic language of biCCC.

I’m not just wasting your time and mine, however; there is a way to combine the flexibility of biCCC with the friendliness of lambda calculus: automatically translate from lambda calculus to biCCC form. The discovery that typed lambda calculus can be interpreted in any CCC is due to Joachim Lambek. See pointers on John Baez’s blog. (Coproducts do not arise in translation unless the source language has a constraint like if-then-else or definition by cases with pattern matching.)

Overview: from lambda expressions to biCCC

We’re going to need a few pieces to complete this story and have it be useful in a language like Haskell:

• a representation of lambda expressions,
• a representation of biCCC expressions,
• a translation of lambda expressions to biCCC, and
• a translation of Haskell to lambda expressions.

This last step (which is actually the first step in turning Haskell into biCCC) is already done by a typical compiler. We start with a syntactically rich language and desugar it into a much smaller lambda calculus. GHC in particular has a small language called “Core”, which is much smaller than the Haskell source language.

I originally intended to convert from Core directly to biCCC form, but I found it difficult to do correctly. Core is dynamically typed, so a type-correct Haskell program can manipulate Core in type-incorrect ways. In other words, a type-correct Haskell program can construct type-incorrect Core. Moreover, Core representations contain an enormous amount of type information, since all type inference has already been done and recorded, so it is tedious to get all of the type information correct and thus likely to get it incorrect. For just this reason, GHC includes an explicit type-checker, “Core Lint”, for catching type inconsistencies (but not their causes) after the fact. While Core Lint is much better than nothing, it is less helpful than static checking, which points to inconsistencies in the source code (of the Core-manipulation).

Because I want static checking of my source code for lambda-to-biCCC conversion, I defined my own alternative to Core, using a generalized algebraic data type (GADT). The first step of translation then is conversion from GHC Core into this GADT.

The source fragments I’ll show below are from the Github project lambda-ccc.

A typeful lambda calculus representation

In Haskell, pair types are usually written “(a,b)”, sums as “Either a b”, and functions as “a → b”. For the categorical generalizations (products, coproducts, and exponentials), I’ll instead use the notation “a × b”, “a + b”, and “a ⇨ b”. (My blogging software typesets some operators differently from what you’ll see in the source code.)

infixl 7 ×
infixl 6 +
infixr 1 ⇨

For reasons to become clearer in future posts, I’ll want a typed representation of types. The data constructors named to reflect the types they construct:

data Ty ∷ * → * where
  Unit ∷ Ty Unit
  (×)  ∷ Ty a → Ty b → Ty (a × b)
  (+)  ∷ Ty a → Ty b → Ty (a + b)
  (⇨)  ∷ Ty a → Ty b → Ty (a ⇨ b)

Note that Ty a is a singleton or empty for every type a. I could instead use promoted data type constructors and singletons.

Next, names and typed variables:

type Name = String
data V a = V Name (Ty a)

Lambda expressions contain binding patterns. For now, we’ll have just the unit pattern, variables, and pair of patterns:

data Pat ∷ * → * where
  UnitPat ∷ Pat Unit
  VarPat  ∷ V a → Pat a
  (:#)    ∷ Pat a → Pat b → Pat (a × b)

Finally, we have lambda expressions, with constructors for variables, constants, application, and abstraction:

infixl 9 :^
data E ∷ * → * where
  Var    ∷ V a → E a
  ConstE ∷ Prim a → Ty a → E a
  (:^)   ∷ E (a ⇨ b) → E a → E b
  Lam    ∷ Pat a → E b → E (a ⇨ b)

The Prim GADT contains typed primitives. The ConstE constructor accompanies a Prim with its specific type, since primitives can be polymorphic.

A typeful biCCC representation

The data type a ↣ b contains biCCC expressions that represent morphisms from a to b:

data (↣) ∷ * → * → * where
  -- Category
  Id      ∷ a ↣ a
  (:∘)    ∷ (b ↣ c) → (a ↣ b) → (a ↣ c)
  -- Products
  Exl     ∷ a × b ↣ a
  Exr     ∷ a × b ↣ b
  (:△)    ∷ (a ↣ b) → (a ↣ c) → (a ↣ b × c)
  -- Coproducts
  Inl     ∷ a ↣ a + b
  Inr     ∷ b ↣ a + b
  (:▽)    ∷ (b ↣ a) → (c ↣ a) → (b + c ↣ a)
  -- Exponentials
  Apply   ∷ (a ⇨ b) × a ↣ b
  Curry   ∷ (a × b ↣ c) → (a ↣ (b ⇨ c))
  Uncurry ∷ (a ↣ (b ⇨ c)) → (a × b ↣ c)
  -- Primitives
  Prim    ∷ Prim (a → b) → (a ↣ b)
  Const   ∷ Prim       b  → (a ↣ b)

The actual representation has some constraints on the type variables involved. I could have used type classes instead of a GADT here, except that the existing classes do not allow polymorphism constraints on the methods. The ConstraintKinds language extension allows instance-specific constraints, but I’ve been unable to work out the details in this case.

I’m not happy with the similarity of Prim and Const. Perhaps there’s a simpler formulation.

Lambda to CCC

We’ll always convert terms of the form λ p → e, and we’ll keep the pattern p and expression e separate:

convert ∷ Pat a → E b → (a ↣ b)

The pattern argument gets built up from patterns appearing in lambdas and serves as a variable binding “context”. To begin, we’ll strip the pattern off of a lambda, eta-expanding if necessary:

toCCC ∷ E (a ⇨ b) → (a ↣ b)
toCCC (Lam p e) = convert p e
toCCC e = toCCC (etaExpand e)

(We could instead begin with a dummy unit pattern/context, giving toCCC the type E c → (() ↣ c).)

The conversion algorithm uses a collection of simple equivalences.

For constants, we have a simple equivalence:

λ p → c ≡ const c

Thus the implementation:

convert _ (ConstE o _) = Const o

For applications, split the expression in two (repeating the context), compute the function and argument parts separately, combine with (△), and then apply:

λ p → u v ≡ apply ∘ ((λ p → u) △ (λ p → v))

The implementation:

convert p (u :^ v) = Apply :∘ (convert p u :△ convert p v)

For lambda expressions, simply curry:

λ p → λ q → e  ≡ curry (λ (p,q) → e)

Assume that there is no variable shadowing, so that p and q have no variables in common. The implementation:

convert p (Lam q e) = Curry (convert (p :# q) e)

Finally, we have to deal with variables. Given λ p → v for a pattern p and variable v appearing in p, either v ≡ p, or p is a pair pattern with v appearing in the left or the right part. To handle these three possibilities, appeal to the following equivalences:

λ v → v     ≡ id
λ (p,q) → e ≡ (λ p → e) ∘ exl  -- if q not free in e
λ (p,q) → e ≡ (λ q → e) ∘ exr  -- if p not free in e

By a pattern not occurring freely, I mean that no variable in the pattern occurs freely.

These properties lead to an implementation:

convert (VarPat u) (Var v) | u ≡ v              = Id
convert (p :# q)   e       | not (q `occurs` e) = convert p e :∘ Exl
convert (p :# q)   e       | not (p `occurs` e) = convert q e :∘ Exr

There are two problems with this code. The first is a performance issue. The recursive convert calls will do considerable redundant work due to the recursive nature of occurs.

To fix this performance problem, handle only λ p → v (variables), and search through the pattern structure only once, returning a Maybe (a ↣ b). The return value is Nothing when v does not occur in p.

convert p (Var v) =
  fromMaybe (error ("convert: unbound variable: " ++ show v)) $
  convertVar p k

If a sub-pattern search succeeds, tack on the ( ∘ Exl) or ( ∘ Exr) using (<$>) (i.e., fmap). Backtrack using mplus.

convertVar ∷ ∀ b a. V b → Pat a → Maybe (a ↣ b)
convertVar u = conv
 where
   conv ∷ Pat c → Maybe (c ↣ b)
   conv (VarPat v) | u ≡ v    = Just Id
                   | otherwise = Nothing
   conv UnitPat  = Nothing
   conv (p :# q) = ((:∘ Exr) <$> conv q) `mplus` ((:∘ Exl) <$> conv p)

(The explicit type quantification and the ScopedTypeVariables language extension relate the b the signatures of convertVar and conv. Note that we’ve solved the problem of redundant occurs testing, eliminating those tests altogether.

The second problem is more troubling: the definitions of convert for Var above do not type-check. Look again at the first try:

convert ∷ Pat a → E b → (a ↣ b)
convert (VarPat u) (Var v) | u ≡ v = Id

The error message:

Could not deduce (b ~ a)
...
Expected type: V a
  Actual type: V b
In the second argument of `(==)', namely `v'
In the expression: u == v

The bug here is that we cannot compare u and v for equality, because they may differ. The definition of convertVar has a similar type error.

Taking care with types

There’s a trick I’ve used in many libraries to handle this situation of wanting to compare for equality two values that may or may not have the same type. For equal values, don’t return simply True, but rather a proof that the types do indeed match. For unequal values, we simply fail to return an equality proof. Thus the comparison operation on V has the following type:

varTyEq ∷ V a → V b → Maybe (a :=: b)

where a :=: b is populated only proofs that a and b are the same type.

The type of type equality proofs is defined in Data.Proof.EQ from the ty package:

data (:=:) ∷ * → * → * where Refl ∷ a :=: a

The Refl constructor is name to suggest the axiom of reflexivity, which says that anything is equal to itself. There are other utilities for commutativity, associativity, and lifting of equality to type constructors.

In fact, this pattern comes up often enough that there’s a type class in the Data.IsTy module of the ty package:

class IsTy f where
  tyEq ∷ f a → f b → Maybe (a :=: b)

With this trick, we can fix our type-incorrect code above. Instead of

convert (VarPat u) (Var v) | u ≡ v = Id

define

convert (VarPat u) (Var v) | Just Refl ← u `tyEq` v = Id

During type-checking, GHC uses the guard (“Just Refl ← u `tyEq` v”) to deduce an additional local constraint to use in type-checking the right-hand side (here Id). That constraint (a ~ b) suffices to make the definition type-correct.

In the same way, we can fix the more efficient implementation:

convertVar ∷ ∀ b a. V b → Pat a → Maybe (a ↣ b)
convertVar u = conv
 where
   conv ∷ Pat c → Maybe (c ↣ b)
   conv (VarPat v) | Just Refl ← v `tyEq` u = Just Id
                   | otherwise              = Nothing
   conv UnitPat  = Nothing
   conv (p :# q) = ((:∘ Exr) <$> conv q) `mplus` ((:∘ Exl) <$> conv p)

Example

To see how conversion works in practice, consider a simple swap function:

swap (a,b) = (b,a)

When reified (as explained in a future post), we get

λ ds → (λ (a,b) → (b,a)) ds

Lambda expressions can be optimized at construction, in which case an $\eta$-reduction would yield the simpler λ (a,b) → (b,a). However, to make the translation more interesting, I’ll leave the lambda term unoptimized.

With the conversion algorithm given above, the (unoptimized) lambda term gets translated into the following:

apply ∘ (curry (apply ∘ (apply ∘ (const (,) △ (id ∘ exr) ∘ exr) △ (id ∘ exl) ∘ exr)) △ id)

Reformatted with line breaks:

  apply
. ( curry (apply ∘ ( apply ∘ (const (,) △ (id ∘ exr) ∘ exr)
                   △ (id ∘ exl) ∘ exr) )
  △ id )

If you squint, you may be able to see how this CCC expression relates to the lambda expression. The “λ ds →” got stripped initially. The remaining application “(λ (a,b) → (b,a)) ds” became apply ∘ (⋯ △ ⋯), where the right “⋯” is id, which came from ds. The left “⋯” has a curry from the “λ (a,b) →” and two applys from the curried application of (,) to b and a. The variables b and a become (id ∘ exr) ∘ exr and (id ∘ exl) ∘ exr, which are paths to b and a in the constructed binding pattern (ds,(a,b)).

I hope this example gives you a feeling for how the lambda-to-CCC translation works in practice, and for the complexity of the result. Fortunately, we can simplify the CCC terms as they’re constructed. For this example, as we’ll see in the next post, we get a much simpler result:

exr △ exl

This combination is common enough that it pretty-prints as

swapP

when CCC desugaring is turned on. (The “P” suffix refers to “product”, to distinguish from coproduct swap.)

Coming up

I’ll close this blog post now to keep it digestible. Upcoming posts will address optimization of biCCC expressions, circuit generation and analysis as biCCCs, and the GHC plugin that handles conversion of Haskell code to biCCC form, among other topics.

Since fall of last year, I’ve been working at Tabula, a Silicon Valley start-up developing an innovative programmable hardware architecture called “Spacetime”, somewhat similar to an FPGA, but much more flexible and efficient. I met the founder, Steve Teig, at a Bay Area Haskell Hackathon in February of 2011. He described his Spacetime architecture, which is based on the geometry of the same name, developed by Hermann Minkowski to elegantly capture Einstein’s theory of special relativity. Within the first 30 seconds or so of hearing what Steve was up to, I knew I wanted to help.

The vision Steve shared with me included not only a better alternative for hardware designers (programmed in hardware languages like Verilog and VHDL), but also a platform for massively parallel execution of software written in a purely functional language. Lately, I’ve been working mainly on this latter aspect, and specifically on the problem of how to compile Haskell. Our plan is to develop the Haskell compiler openly and encourage collaboration. If anything you see in this blog series interests you, and especially if have advice or you’d like to collaborate on the project, please let me know.

In my next series of blog posts, I’ll describe some of the technical ideas I’ve been working with for compiling Haskell for massively parallel execution. For now, I want to introduce a central idea I’m using to approach the problem.

Lambda calculus and cartesian closed categories

I’m used to thinking of the typed lambda calculi as languages for describing functions and other mathematical values. For instance, if the type of an expression e is Bool → Bool, then the meaning of e is a function from Booleans to Booleans. (In non-strict pure languages like Haskell, both Boolean types include ⊥. In hypothetically pure strict languages, the range is extend to include ⊥, but the domain isn’t.)

However, there are other ways to interpret typed lambda-calculi.

You may have heard of “cartesian closed categories” (CCCs). CCC is an abstraction having a small vocabulary with associated laws:

• The “category” part means we have a notion of “morphisms” (or “arrows”) each having a domain and codomain “object”. There is an identity morphism for and associative composition operator. If this description of morphisms and objects sounds like functions and types (or sets), it’s because functions and types are one example, with id and (∘).
• The “cartesian” part means that we have products, with projection functions and an operator to combine two functions into a pair-producing function. For Haskell functions, these operations are fst and snd, together with (&&&) from Control.Arrow.
• The “closed” part means that we have a way to represent morphisms via objects, referred to as “exponentials”. The corresponding operations are curry, uncurry, and apply. Since Haskell is a higher-order language, these exponential objects are simply (first class) functions.

A wonderful thing about the CCC interface is that it suffices to translate any lambda expression, as discovered by Joachim Lambek. In other words, lambda expressions can be systematically translated into the CCC vocabulary. Any (law-abiding) interpretation of that vocabulary is thus an interpretation of the lambda calculus.

Besides intellectual curiosity, why might one care about interpreting lambda expressions in terms of CCCs other than the one we usually think of for functional programs? I got interested because I’ve been thinking about how to compile Haskell programs to “circuits”, both the standard static kind and more dynamic variants. Since Haskell is a typed lambda calculus, if we can formulate circuits as a CCC, we’ll have our Haskell-to-circuit compiler. Other interpretations enable analysis of timing and demand propagation (including strictness).

Some future topics

• Converting lambda expressions to CCC form.
• Optimizing CCC expressions.
• Plugging into GHC, to convert from Haskell source to CCC.
• Applications of this translation, including the following:
  • Circuits
  • Timing analysis
  • Strictness/demand analysis
  • Type simplification (normalization)

Why is matrix multiplication defined so very differently from matrix addition? If we didn’t know these procedures, could we derive them from first principles? What might those principles be?

This post gives a simple semantic model for matrices and then uses it to systematically derive the implementations that we call matrix addition and multiplication. The development illustrates what I call “denotational design”, particularly with type class morphisms. On the way, I give a somewhat unusual formulation of matrices and accompanying definition of matrix “multiplication”.

For more details, see the linear-map-gadt source code.

Edits:

• 2012–12–17: Replaced lost $B$ entries in description of matrix addition. Thanks to Travis Cardwell.
• 2012–12018: Added note about math/browser compatibility.

Note: I’m using MathML for the math below, which appears to work well on Firefox but on neither Safari nor Chrome. I use Pandoc to generate the HTML+MathML from markdown+lhs+LaTeX. There’s probably a workaround using different Pandoc settings and requiring some tweaks to my WordPress installation. If anyone knows how (especially the WordPress end), I’d appreciate some pointers.

Matrices

For now, I’ll write matrices in the usual form: $$\left(\begin{array}{ccc}{A}_{11}& \cdots & A_{1m}\\ ⋮& \ddots & ⋮\\ A_{n1}& \cdots & A_{nm}\end{array}\right)$$

Addition

To add two matrices, we add their corresponding components. If $$A=\left(\begin{array}{ccc}{A}_{11}& \cdots & A_{1m}\\ ⋮& \ddots & ⋮\\ A_{n1}& \cdots & A_{nm}\end{array}\right)\mathrm{\text{and}}B=\left(\begin{array}{ccc}{B}_{11}& \cdots & B_{1m}\\ ⋮& \ddots & ⋮\\ B_{n1}& \cdots & B_{nm}\end{array}\right),$$ then $$A+B=\left(\begin{array}{ccc}{A}_{11}+B_{11}& \cdots & A_{1m}+B_{1m}\\ ⋮& \ddots & ⋮\\ A_{n1}+B_{n1}& \cdots & A_{nm}+B_{nm}\end{array}\right).$$ More succinctly, $$(A+B{)}_{ij}=A_{ij}+B_{ij}.$$

Multiplication

Multiplication, on the other hand, works quite differently. If $$A=\left(\begin{array}{ccc}{A}_{11}& \cdots & A_{1m}\\ ⋮& \ddots & ⋮\\ A_{n1}& \cdots & A_{nm}\end{array}\right)\mathrm{\text{and}}B=\left(\begin{array}{ccc}{B}_{11}& \cdots & B_{1p}\\ ⋮& \ddots & ⋮\\ B_{m1}& \cdots & B_{mp}\end{array}\right),$$ then $$(A\bullet B{)}_{ij}=\sum _{k=1}^m{A}_{ik}\cdot B_{kj}.$$ This time, we form the dot product of each $A$ row and $B$ column.

Why are these two matrix operations defined so differently? Perhaps these two operations are implementations of more fundamental specifications. If so, then making those specifications explicit could lead us to clear and compelling explanations of matrix addition and multiplication.

Transforming vectors

Simplifying from matrix multiplication, we have transformation of a vector by a matrix. If $$A=\left(\begin{array}{ccc}{A}_{11}& \cdots & A_{1m}\\ ⋮& \ddots & ⋮\\ A_{n1}& \cdots & A_{nm}\end{array}\right)\mathrm{\text{and}}x=\left(\begin{array}{c}{x}_1\\ ⋮\\ x_m\end{array}\right),$$ then $$A\cdot x=\left(\begin{array}{ccccc}{A}_{11}\cdot x_1& +& \cdots & +& A_{1m}\cdot x_m\\ ⋮& & \ddots & & ⋮\\ A_{n1}\cdot x_1& +& \cdots & +& A_{nm}\cdot x_m\end{array}\right)$$ More succinctly, $$(A\cdot x{)}_i=\sum _{k=1}^m{A}_{ik}\cdot x_k.$$

What’s it all about?

We can interpret matrices as transformations. Matrix addition then adds transformations:

$$(A+B)x=Ax+Bx$$

Matrix “multiplication” composes transformations:

$$(A\bullet B)x=A(Bx)$$

What kinds of transformations?

Linear transformations

Matrices represent linear transformations. To say that a transformation (or “function” or “map”) $f$ is “linear” means that $f$ preserves the structure of addition and scalar multiplication. In other words, $$\begin{array}{ccc}\hfill f(x+y)& \hfill =\hfill & fx+fy\hfill \\ \hfill f(c\cdot x)& \hfill =\hfill & c\cdot fx\hfill \end{array}$$ Equivalently, $f$ preserves all linear combinations: $$f(c_1\cdot x_1+\cdots +c_m\cdot x_m)=c_1\cdot f{x}_1+\cdots +c_m\cdot f{x}_m$$

What does it mean to say that “matrices represent linear transformations”? As we saw in the previous section, we can use a matrix to transform a vector. Our semantic function will exactly be this use, i.e., the meaning of matrix is as a function (map) from vectors to vectors. Moreover, these functions will satisfy the linearity properties above.

Representation

For simplicity, I’m going structure matrices in a unconventional way. Instead of a rectangular arrangement of numbers, use the following generalized algebraic data type (GADT):

data a ⊸ b where
  Dot   ∷ InnerSpace b ⇒
          b → (b ⊸ Scalar b)
  (:&&) ∷ VS3 a c d ⇒  -- vector spaces with same scalar field
          (a ⊸ c) → (a ⊸ d) → (a ⊸ c × d)

I’m using the notation “c × d” in place of the usual “(c,d)”. Precedences are such that “×” binds more tightly than “⊸”, which binds more tightly than “→”.

This definition builds on the VectorSpace class, with its associated Scalar type and InnerSpace subclass. Using VectorSpace is overkill for linear maps. It suffices to use modules over semirings, which means that we don’t assume multiplicative or additive inverses. The more general setting enables many more useful applications than vector spaces do, some of which I will describe in future posts.

The idea here is that a linear map results in either (a) a scalar, in which case it’s equivalent to dot v (partially applied dot product) for some v, or (b) a product, in which case it can be decomposed into two linear maps with simpler range types. Each row in a conventional matrix corresponds to Dot v for some vector v, and the stacking of rows corresponds to nested applications of (:&&).

Semantics

The semantic function, apply, interprets a representation of a linear map as a function (satisfying linearity):

apply ∷ (a ⊸ b) → (a → b)
apply (Dot b)   = dot b
apply (f :&& g) = apply f &&& apply g

where, (&&&) is from Control.Arrow.

(&&&) ∷ Arrow (↝) ⇒ (a ↝ b) → (a ↝ c) → (a ↝ (b,c))

For functions,

(f &&& g) a = (f a, g a)

Functions, linearity, and multilinearity

Functions form a vector space, with scaling and addition defined “pointwise”. Instances from the vector-space package:

instance AdditiveGroup v ⇒ AdditiveGroup (a → v) where
  zeroV   = pure   zeroV
  (^+^)   = liftA2 (^+^)
  negateV = fmap   negateV

instance VectorSpace v ⇒ VectorSpace (a → v) where
  type Scalar (a → v) = a → Scalar v
  (*^) s = fmap (s *^)

I wrote the definitions in this form to fit a template for applicative functors in general. Inlining the definitions of pure, liftA2, and fmap on functions, we get the following equivalent instances:

instance AdditiveGroup v ⇒ AdditiveGroup (a → v) where
  zeroV     = λ _ → zeroV
  f ^+^ g   = λ a → f a ^+^ g a
  negateV f = λ a → negateV (f a)

instance VectorSpace v ⇒ VectorSpace (a → v) where
  type Scalar (a → v) = a → Scalar v
  s *^ f = λ a → s *^ f a

In math, we usually say that dot product is “bilinear”, or “linear in each argument”, i.e.,

dot (s *^ u,v) ≡ s *^ dot (u,v)
dot (u ^+^ w, v) ≡ dot (u,v) ^+^ dot (w,v)

Similarly for the second argument:

dot (u,s *^ v) ≡ s *^ dot (u,v)
dot (u, v ^+^ w) ≡ dot (u,v) ^+^ dot (u,w)

Now recast the first of these properties in a curried form:

dot (s *^ u) v ≡ s *^ dot u v

i.e.,

dot (s *^ u)
 ≡ {- η-expand -}
λ v → dot (s *^ u) v
 ≡ {- "bilinearity" -}
λ v → s *^ dot u v
 ≡ {- (*^) on functions -}
λ v → (s *^ dot u) v
 ≡ {- η-contract -}
s *^ dot u

Likewise,

dot (u ^+^ v)
 ≡ {- η-expand -}
λ w → dot (u ^+^ v) w
 ≡ {- "bilinearity" -}
λ w → dot u w ^+^ dot v w
 ≡ {- (^+^) on functions -}
dot u ^+^ dot v

Thus, when “bilinearity” is recast in terms of curried functions, it becomes just linearity. (The same reasoning applies more generally to multilinearity.)

Note that we could also define function addition as follows:

f ^+^ g = add ∘ (f &&& g)

where

add = uncurry (^+^)

This uncurried form will come in handy in derivations below.

Deriving matrix operations

Addition

We’ll add two linear maps using the (^+^) operation from Data.AdditiveGroup.

(^+^) ∷ (a ⊸ b) → (a ⊸ b) → (a ⊸ b)

Following the principle of semantic type class morphisms, the specification simply says that the meaning of the sum is the sum of the meanings:

apply (f ^+^ g) ≡ apply f ^+^ apply g

which is half of the definition of “linearity” for apply.

The game plan (as always) is to use the semantic specification to derive (or “calculate”) a correct implementation of each operation. For addition, this goal means we want to come up with a definition like

f ^+^ g = <rhs>

where <rhs> is some expression in terms of f and g whose meaning is the same as the meaning as f ^+^ g, i.e., where

apply (f ^+^ g) ≡ apply <rhs>

Since Haskell has convenient pattern matching, we’ll use it for our definition of (^+^) above. Addition has two arguments, and our data type has two constructors, there are at most four different cases to consider.

First, add Dot and Dot. The specification

apply (f ^+^ g) ≡ apply f ^+^ apply g

specializes to

apply (Dot b ^+^ Dot c) ≡ apply (Dot b) ^+^ apply (Dot c)

Now simplify the right-hand side (RHS):

apply (Dot b) ^+^ apply (Dot c)
 ≡ {- apply definition -}
dot b ^+^ dot c
 ≡ {- (bi)linearity of dot, as described above -}
dot (b ^+^ c)
 ≡ {- apply definition -}
apply (Dot (b ^+^ c))

So our specialized specification becomes

apply (Dot b ^+^ Dot c) ≡ apply (Dot (b ^+^ c))

which is implied by

Dot b ^+^ Dot c ≡ Dot (b ^+^ c)

and easily satisfied by the following partial definition (replacing “≡” by “=”):

Dot b ^+^ Dot c = Dot (b ^+^ c)

Now consider the case of addition with two (:&&) constructors:

The specification specializes to

apply ((f :&& g) ^+^ (h :&& k)) ≡ apply (f :&& g) ^+^ apply (h :&& k)

As with Dot, simplify the RHS:

apply (f :&& g) ^+^ apply (h :&& k)
 ≡ {- apply definition -}
(apply f &&& apply g) ^+^ (apply h &&& apply k)
 ≡ {- See below -}
(apply f ^+^ apply h) &&& (apply g ^+^ apply k)
 ≡ {- induction -}
apply (f ^+^ h) &&& apply (g ^+^ k)
 ≡ {- apply definition -}
apply ((f ^+^ h) :&& (g ^+^ k))

I used the following property (on functions):

(f &&& g) ^+^ (h &&& k) ≡ (f ^+^ h) &&& (g ^+^ k)

Proof:

(f &&& g) ^+^ (h &&& k)
 ≡ {- η-expand -}
λ x → ((f &&& g) ^+^ (h &&& k)) x
 ≡ {- (&&&) definition for functions -}
λ x → (f x, g x) ^+^ (h x, k x)
 ≡ {- (^+^) definition for pairs -}
λ x → (f x ^+^ h x, g x ^+^ k x)
 ≡ {- (^+^) definition for functions -}
λ x → ((f ^+^ h) x, (g ^+^ k) x)
 ≡ {- (&&&) definition for functions -}
(f ^+^ h) &&& (g ^+^ k)

The specification becomes

apply ((f :&& g) ^+^ (h :&& k)) ≡ apply ((f ^+^ h) :&& (g ^+^ k))

which is easily satisfied by the following partial definition

(f :&& g) ^+^ (h :&& k) = (f ^+^ h) :&& (g ^+^ k)

The other two cases are (a) Dot and (:&&), and (b) (:&&) and Dot, but they don’t type-check (assuming that pairs are not scalars).

Composing linear maps

I’ll write linear map composition as “g ∘ f”, with type

(∘) ∷ (b ⊸ c) → (a ⊸ b) → (a ⊸ c)

This notation is thanks to a Category instance, which depends on a generalized Category class that uses the recent ConstraintKinds language extension. (See the source code.)

Following the semantic type class morphism principle again, the specification says that the meaning of the composition is the composition of the meanings:

apply (g ∘ f) ≡ apply g ∘ apply f

In the following, note that the ∘ operator binds more tightly than &&&, so f ∘ h &&& g ∘ h means (f ∘ h) &&& (g ∘ h).

Derivation

Again, since there are two constructors, we have four possible cases cases. We can handle two of these cases together, namely (:&&) and anything. The specification:

apply ((f :&& g) ∘ h) ≡ apply (f :&& g) ∘ apply h

Reasoning proceeds as above, simplifying the RHS of the constructor-specialized specification.

Simplify the RHS:

apply (f :&& g) ∘ apply h
 ≡ {- apply definition -}
(apply f &&& apply g) ∘ apply h
 ≡ {- see below -}
apply f ∘ apply h &&& apply g ∘ apply h
 ≡ {- induction -}
apply (f ∘ h) &&& apply (g ∘ h)
 ≡ {- apply definition -}
apply (f ∘ h :&& g ∘ h)

This simplification uses the following property of functions:

(p &&& q) ∘ r ≡ p ∘ r &&& q ∘ r

Sufficient definition:

(f :&& g) ∘ h = f ∘ h :&& g ∘ h

We have two more cases, specified as follows:

apply (Dot c ∘ Dot b) ≡ apply (Dot c) ∘ apply (Dot b)

apply (Dot c ∘ (f :&& g)) ≡ apply (Dot c) ∘ apply (f :&& g)

Based on types, c must be a scalar in the first case and a pair in the second. (Dot b produces a scalar, while f :&& g produces a pair.) Thus, we can write these two cases more specifically:

apply (Dot s ∘ Dot b) ≡ apply (Dot s) ∘ apply (Dot b)

apply (Dot (a,b) ∘ (f :&& g)) ≡ apply (Dot (a,b)) ∘ apply (f :&& g)

In the derivation, I won’t spell out as many details as before. Simplify the RHSs:

  apply (Dot s) ∘ apply (Dot b)
≡ dot s ∘ dot b
≡ dot (s *^ b)
≡ apply (Dot (s *^ b))

  apply (Dot (a,b)) ∘ apply (f :&& g)
≡ dot (a,b) ∘ (apply f &&& apply g)
≡ add ∘ (dot a ∘ apply f &&& dot b ∘ apply g)
≡ dot a ∘ apply f ^+^ dot b ∘ apply g
≡ apply (Dot a ∘ f ^+^ Dot b ∘ g)

I’ve used the following properties of functions:

dot (a,b)             ≡ add ∘ (dot a *** dot b)

(r *** s) ∘ (p &&& q) ≡ r ∘ p &&& s ∘ q

add ∘ (p &&& q)       ≡ p ^+^ q

apply (f ^+^ g)       ≡ apply f ^+^ apply g

Implementation:

 Dot s     ∘ Dot b     = Dot (s *^ b)
 Dot (a,b) ∘ (f :&& g) = Dot a ∘ f ^+^ Dot b ∘ g

Cross products

Another Arrow operation handy for linear maps is the parallel composition (product):

(***) ∷ (a ⊸ c) → (b ⊸ d) → (a × b ⊸ c × d)

The specification says that apply distributes over (***). In other words, the meaning of the product is the product of the meanings.

apply (f *** g) = apply f *** apply g

Where, on functions,

p *** q = λ (a,b) → (p a, q b)
        ≡ p ∘ fst &&& q ∘ snd

Simplify the specifications RHS:

  apply f *** apply g
≡ apply f ∘ fst &&& apply g ∘ snd

If we knew how to represent fst and snd via our linear map constructors, we’d be nearly done. Instead, let’s suppose we have the following functions.

compFst ∷ VS3 a b c ⇒ a ⊸ c → a × b ⊸ c
compSnd ∷ VS3 a b c ⇒ b ⊸ c → a × b ⊸ c

specified as follows:

apply (compFst f) ≡ apply f ∘ fst
apply (compSnd g) ≡ apply g ∘ snd

With these two functions (to be defined) in hand, let’s try again.

  apply f *** apply g
≡ apply f ∘ fst &&& apply g ∘ snd
≡ apply (compFst f) &&& apply (compSnd g)
≡ apply (compFst f :&& compSnd g)

Composing with fst and snd

I’ll elide even more of the derivation this time, focusing reasoning on the meanings. Relating to the representation is left as an exercise. The key steps in the derivation:

dot a     ∘ fst ≡ dot (a,0)

(f &&& g) ∘ fst ≡ f ∘ fst &&& g ∘ fst

dot b     ∘ snd ≡ dot (0,b)

(f &&& g) ∘ snd ≡ f ∘ snd &&& g ∘ snd

Implementation:

compFst (Dot a)   = Dot (a,zeroV)
compFst (f :&& g) = compFst f &&& compFst g

compSnd (Dot b)   = Dot (zeroV,b)
compSnd (f :&& g) = compSnd f &&& compSnd g

where zeroV is the zero vector.

Given compFst and compSnd, we can implement fst and snd as linear maps simply as compFst id and compSnd id, where id is the (polymorphic) identity linear map.

Reflections

This post reflects an approach to programming that I apply wherever I’m able. As a summary:

• Look for an elegant what behind a familiar how.
• Define a semantic function for each data type.
• Derive a correct implementation from the semantics.

You can find more examples of this methodology elsewhere in this blog and in the paper Denotational design with type class morphisms.