Functional programming abounds with recursively defined data types. We often draw these structured values as trees with the root at the top and the leaves at the bottom. Lazy functional programming allows values (structures) of these types to be “bottomless”, meaning we can descend forever. There are many examples of how supporting such values gives an enormous boost to modularity. (See, e.g., John Hughes’s classic paper Why Functional Programming Matters.) We usually refer to these values as “infinite”, but I’d like to suggest “bottomless” as a more specific alternative, and to point out a limitation that perhaps is not widely noticed.

Although we can descend infinitely in lazy functional programming, we can only ascend finitely. If I’m traversing a lazy list, there may be infinitely many elements on my right (yet to be visited) but only finitely many on my left (already visited). While traversing a tree, there may be infinite paths below but only a finite one above (leading to my current position).

In other words, our data is bottomless, but not topless. What would it be like to go beyond our usual merely uni-infinite data and program with bi-infinite data instead? With data that is both bottomless and topless?

In what sense are our current lazy data types only uni-infinite, i.e., “topped” rather than topless. What does it even mean to ascend in a data structure, when all of the pointers are downward (from parent to child)? I’m thinking of recursively defined functions, in which each descension is a recursive call (passing in a structure’s component), and each ascension is a return. The topped nature of data is reflected in the finiteness of the call stack at every point.

What keeps our call stacks finite? I guess that the stack starts out empty, and each deepening step (function call) takes some time. In contrast, the data depth can start out infinite.

To get topless data, start with an infinite stack.

To apply this idea in an existing language, say Haskell, start by reifying the control stack into a lazy data structure of some sort. For instance, a lazy list (stream). Or a zipper! As pointed out in Another angle on zippers, these two ideas are very close together. A zipper can be represented as a list of derivatives (one-hole contexts), together with a sub-structure of focus:

type Context t = [Der (PF t) t]

type Zipper t = (Context t, t)

This Context type uses a list to allow for topped data. For purely topless types (as in the example below), we can use a more precise type of infinite-only lists:

type Context t = Stream (Der (PF t) t)

where Stream comes from Wouter Swierstra‘s Stream package:

data Stream a = Cons a (Stream a)

Using Stream, we can eliminate the only failure mode in the zipper operations of Another angle on zippers, namely going up with an empty context.

Now let’s look at an example of a purely topless data type.

Representing images

I’ve noticed functional programming’s bottomless vs topless disparity over the years when thinking about functional imagery. I like formulating imagery as functions over continuous and infinite space, i.e., infinite in detail and infinite in extent. These properties make images more simply and generally composable than the usual implementation-oriented approach to images (discrete and finite). (Ditto for behavior, as functions of continuous and infinite time.)

An easy way to implement the function-of-space model of images is directly as functions of space, i.e., use the computable functions from our programming language to represent the mathematical functions in the semantic model. In Pan and a few following systems, I used a variant of this simple implementation approach, using Haskell functions that generate expressions to be compiled into highly optimized low-level code.

Representing functions as functions has the benefit of simplicity. Ironically, however, functional programming penalizes the use of functions as a representation. Other than functions, data representations cache their components. For instance, if I extract the second element of a triple twice, that element will be computed only once. In contrast, if I apply a function twice to the second value in a three-value domain (e.g., Cold, Medium, Hot), then the result will be computed twice, not once.

One way to avoid this penalty is memoization, which is the conversion of functions into data structures. However, I think memoization isn’t quite what I’m looking for to implement imagery or behavior. In those cases, the function domains are continuous, and the likelihood of a repeated sampling at exactly the same domain points is vanishingly small. Well, that statement isn’t true, as it assumes uniformly random sampling over time & space, but often some repeated sampling occurs by construction, and caching has been useful in function-based FRP implementations.

For imagery, I like to pan, zoom, and rotate interactively. The simplest implementation model I know samples an image once for each pixel. A variation, e.g. used in Pan and Pajama, is to sample several times per pixel for progressive, stochastic anti-aliasing. One could perhaps go even further and use something like exact numeric integration to anti-alias perfectly.

Being function-centered, these implementation styles have tremendous waste, computing each display frame (sampling in discrete time & space) from scratch, reusing no sampling work from previous frames, even though the resulting images are very similar while interactively perusing. Modern graphics hardware can pan, zoom, and rotate (PZR) discrete finite samplings (“bitmaps”) with modest filtering very fast.

I want to exploit that hardware without compromising the simple denotational model of continuous & discrete images. An idea I’ve been playing with is to use an image pyramid, which is a stack of progressively higher resolution samplings, as in hardware mip-maps. Those samplings can be organized into an infinite quadtree. Each quadtree level has a texture map (bitmap stored in graphics memory), and all texture maps have the same of samples (pixels), e.g., 256 × 256. In addition to a texture map, each quadtree has four subtrees, corresponding to four spatial subquadrants. Abstracting out the texture map, we might write

data Quadtree a = Quadtree a (Quad (Quadtree a))

type Quad t = (t,t,t,t)

It’s fairly easy to write a lazy functional recursive renderer that converts a function-style image into a quadtree. The recursive argument to this renderer is a square region of space. To display such a tree interactively, at each frame pick a level that suits the current zoom factor, e.g., so that one sample corresponds roughly to one pixel. Rendering a single frame involves choosing a subset of the quadtree’s texture maps and displaying them with hardware-accelerated adjustments for PZR.

As an important optimization, reject any subtrees that are outside of the current view window, given the current view transformation (pan vector, zoom factor, and rotation angle). Thanks to laziness, this structure will get computed and filled in only as it’s accessed. Warren Burton explored lazy functional quadtrees in the 1980s. See, e.g., Functional programming and quadtrees by F. W. Burton and J. G. Kollias in IEEE Software 6(1):90-97, January 1989.

There’s a problem with this idea of using quadtrees for infinite continuous images. The image representation captures the continuous aspect but not the infinite aspect. Or more precisely, it captures infinite resolution but not infinite extent. Descending in a spatial quadtree doubles the resolution (in each direction) and halves the extent. Conversely, ascending halves the resolution and doubles the extent. To find a collection of texture maps of a desired resolution and spatial region, ascend high enough to encompass the region, and then descend low enough to get the required resolution.

So, just as bottomlessness gives rise to infinite resolution, toplessness would give rise to infinite extent.

For functions of continuous & infinite time, the domain is 1D instead of 2D, so replace quadtrees and their zippers by binary trees and their zippers. (See Sequences, segments, and signals for related remarks.)

Aside: thought tools

I have a prototype implementation of the lazy quadtree scheme in Objective C for the iPhone OS. Given the clumsiness of Objective C for functional programming, and particularly the language’s weak compile-time typing and lack of parametric polymorphism, I switched to Haskell to work out the zipper aspect.

Once I was writing Haskell, the imperative machinery faded from view and I couldn’t help but start seeing essential patterns. Tinkering with those patterns led me to new insights, including the ones described in my recent series of posts on higher-order types, derivatives, and zippers. I am grateful to have this higher-order lazy language with rich static typing as a thought tool, to help me gain insights and weed out my mistakes. It’s a bonus to me that the language is executable as well.

Conclusion

This idea of topless data (with its infinite call stacks) has been bumping around in my mind for 15 years or so. It was probably inspired in part by Mitch Wand’s 1980 paper Continuation-Based Program Transformation Strategies, which I read in grad school. That article is on my short list of research writings that have most influenced me. If you haven’t read it, or haven’t read it recently, please do.

In finishing this blog post, I’m a little uncomfortable with the fuzziness of what I’ve presented. Buckminster Fuller said “I call intuition cosmic fishing. You feel a nibble, then you’ve got to hook the fish.” Topless data has been a helpful intuition for me. I’m not sure I’ve quite hooked the fish, but perhaps you’ll catch something useful to you in this post anyway.

The zipper is an efficient and elegant data structure for purely functional editing of tree-like data structures, first published by Gérard Huet. Zippers maintain a location of focus in a tree and support navigation operations (up, down, left, right) and editing (replace current focus).

The original zipper type and operations are customized for a single type, but it’s not hard to see how to adapt to other tree-like types, and hence to regular data types. There have been many follow-up papers to The Zipper, including a polytypic version in the paper Type-indexed data types.

All of the zipper adaptations and generalizations I’ve seen so far maintain the original navigation interface. In this post, I propose an alternative interface that appears to significantly simplify matters. There are only two navigation functions instead of four, and each of the two is specified and implemented via a fairly simple one-liner.

I haven’t used this new zipper formulation in an application yet, so I do not know whether some usefulness has been lost in simplifying the interface.

The code in this blog post is taken from the Haskell library functor-combo and completes the Holey type class introduced in Differentiation of higher-order types.

Edits:

• 2010-07-29: Removed some stray Just applications in up definitions. (Thanks, illissius.)
• 2010-07-29: Augmented my complicated definition of tweak2 with a much simpler version from Sjoerd Visscher.
• 2010-07-29: Replaced fmap (first (:ds')) with (fmap.first) (:ds') in down definitions. (Thanks, Sjoerd.)

Extraction

The post Differentiation of higher-order types gave part of a type class for one-hole contexts (functor derivatives) and the filling of those contexts:

class Functor f ⇒ Holey f where
  type Der f :: * → *
  fillC :: Der f a → a → f a

The arguments of fillC correspond roughly to the components of what Gérard Huet called a “location”, namely context and something to fill the context:

type Loc f a = (Der f a, a)

So an alternative hole-filling interface is

fill :: Holey f ⇒ Loc f a → f a
fill = uncurry fillC

Now consider a reverse operation, a kind of extraction:

guess1 :: f a → Loc f a

There’s an awkward problem here. What if f a has more than one possible hole, or has no hole at all? If more than one, then which do we pick? Perhaps the left-most. If none, then we might want to have a failure representation, e.g.,

guess2 :: f a → Maybe (Loc f a)

To handle the more-than-one possibility, we could add another method for traversing the various extractions, like the go_right operation in The Zipper, section 2.2. I don’t know what changes we’d have to make to the Loc type.

We could instead use a list of possible extractions.

guess3 :: f a → [Loc f a]

Why a list? I guess because it’s in our habitual functional toolbox, and it covers any number of alternative extracted locations. On the other hand, our toolbox is growing, and sometimes list isn’t the best functor for the job. For instance, we might use a finger tree, which has better performance for some sequence operations.

Or we could a functor closer at hand, namely f itself.

class Functor f ⇒ Holey f where
  type Der f :: * → *
  fillC   :: Der f a → a → f a
  extract :: f a → f (Loc f a)

For instance, when f ≡ [], extract returns a list of extractions; and when f ≡ Id :*: Id, extract returns a pair of extractions.

How to extract

A constant functor has void derivative. Extraction yields another constant structure, with the same data but a different type:

instance Holey (Const x) where
  type Der (Const x) = Void
  fillC = voidF
  extract (Const x) = Const x

The identity functor has exactly one opportunity for a hole, leaving no information behind:

instance Holey Id where
  type Der Id = Unit
  fillC (Const ()) = Id
  extract (Id a) = Id (Const (), a)

The definitions of Der and fillC above and below are lifted directly from Differentiation of higher-order types.

For sums, there are two cases: InL fa, InR ga :: (f :+: g) a. Starting with the first case:

InL fa :: (f :+: g) a

fa :: f a

extract fa :: f (Loc f a)

           :: f (Der f a, a)

(fmap.first) InL (extract fa) :: f ((Der f :+: Der g) a, a)

                              :: f ((Der (f :+: g) a), a)

See Semantic editor combinators for an explanation of (fmap.first) friends. Continuing, apply the definition of Der on sums:

InL ((fmap.first) InL (extract fa)) :: (f :+: g) ((Der (f :+: g) a), a)

                                    :: (f :+: g) (Loc (f :+: g) a)

The two steps that introduce g are motivated by the required type of extract. Similarly, for the second case:

InR ((fmap.first) InR (extract ga)) :: (f :+: g) (Loc (f :+: g) a)

So,

instance (Holey f, Holey g) ⇒ Holey (f :+: g) where
  type Der (f :+: g) = Der f :+: Der g
  fillC (InL df) = InL ∘ fillC df
  fillC (InR df) = InR ∘ fillC df
  extract (InL fa) = InL ((fmap.first) InL (extract fa))
  extract (InR ga) = InR ((fmap.first) InR (extract ga))

For products, recall the derivative type:

  type Der (f :*: g) = Der f :*: g  :+:  f :*: Der g

To extract from a product, we extract from either component and then pair with the other component. The form of an argument to extract is

fa :*: ga :: (f :*: g) a

Again, start with the left part:

fa :: f a

extract fa :: f (Loc f a)
           :: f (Der f a, a)

(fmap.first) (:*: ga) (extract fa) :: f ((Der f :*: g) a, a)

(fmap.first) (InL ∘ (:*: ga)) (extract fa)
  :: f (((Der f :*: g) :+: (f :*: Der g)) a, a)

  :: f ((Der (f :*: g)) a, a)

Similarly, for the second component,

(fmap.first) (InR ∘ (fa :*:)) (extract ga)
  :: g ((Der (f :*: g)) a, a)

Combining the two extraction routes:

(fmap.first) (InL ∘ (:*: ga)) (extract fa) :*:
(fmap.first) (InR ∘ (fa :*:)) (extract ga)
  :: (f :*: g) (Der (f :*: g) a, a)

So,

instance (Holey f, Holey g) ⇒ Holey (f :*: g) where
  type Der (f :*: g) = Der f :*: g  :+:  f :*: Der g
  fillC (InL (dfa :*:  ga)) = (:*: ga) ∘ fillC dfa
  fillC (InR ( fa :*: dga)) = (fa :*:) ∘ fillC dga
  extract (fa :*: ga) = 
    (fmap.first) (InL ∘ (:*: ga)) (extract fa) :*:
    (fmap.first) (InR ∘ (fa :*:)) (extract ga)

Finally, the chain rule, for functor composition:

type Der (g :. f) = Der g :. f  :*:  Der f

A value of type (g :. f) a has form O gfa, where gfa :: g (f a). To extract:

• form all g-extractions, yielding values of type fa :: f a and their contexts of type Der g (f a);
• form all f-extractions of each such fa, yielding values of type a and their contexts of type Der f a; and
• reassemble these pieces into the shape determined by Der (g :. f).

Let’s go:

gfa :: g (f a)

extract gfa :: g (Loc g (f a))

            :: g (Der g (f a), f a)

fmap (second extract) (extract gfa)
  :: g (Der g (f a), f (Loc f a))

Continuing, the following lemmas come in handy.

tweak2 :: Functor f ⇒
          (dg (f a), f (df a, a)) → f (((dg :. f) :*: df) a, a)
tweak2 = (fmap.first) chainRule ∘ tweak1

tweak1 :: Functor f ⇒
          (dg (fa), f (dfa, a)) → f ((dg (fa), dfa), a)
tweak1 = fmap lassoc ∘ squishP

squishP :: Functor f ⇒ (a, f b) → f (a,b)
squishP (a,fb) = fmap (a,) fb

chainRule :: (dg (f a), df a) → ((dg :. f) :*: df) a
chainRule (dgfa, dfa) = O dgfa :*: dfa

lassoc :: (p,(q,r)) → ((p,q),r)
lassoc    (p,(q,r)) =  ((p,q),r)

Edit: Sjoerd Visscher found a much simpler form to replace the previous group of definitions:

tweak2 (dgfa, fl) = (fmap.first) (O dgfa :*:) fl

More specifically,

tweak2 :: Functor f => (Der g (f a), f (Loc f a))
                    -> f (((Der g :. f) :*: Der f) a, a)
       :: Functor f => (Der g (f a), f (Loc f a))
                    -> f (Der (g :. f) a, a)
       :: Functor f => (Der g (f a), f (Loc f a))
                    -> f (Loc (g :. f) a)

This lemma gives just what we need to tweak the inner extraction:

fmap (tweak2 ∘ second extract) (extract gfa) :: g (f (Loc (g :. f) a))

So

extractGF :: (Holey f, Holey g) ⇒
             g (f a) → g (f (Loc (g :. f) a))
extractGF = fmap (tweak2 ∘ second extract) ∘ extract

and

instance (Holey f, Holey g) ⇒ Holey (g :. f) where
  type Der (g :.  f) = Der g :. f  :*:  Der f
  fillC (O dgfa :*: dfa) = O ∘ fillC dgfa ∘ fillC dfa
  extract = inO extractGF

where inO is from Control.Compose, and is defined using the ideas from Prettier functions for wrapping and wrapping and the notational improvement from Matt Hellige’s Pointless fun.

-- | Apply a unary function within the 'O' constructor.
inO :: (g (f a) → g' (f' a')) → ((g :. f) a → (g' :. f') a')
inO = unO ~> O

infixr 1 ~>
-- | Add pre- and post processing
(~>) :: (a' → a) → (b → b') → ((a → b) → (a' → b'))
(f ~> h) g = h ∘ g ∘ f

In case you’re wondering, these definitions did not come to me effortlessly. I sweated through the derivation, guided always by my intuition and the necessary types, as determined by the shape of Der (g :. f). The type-checker helped me get from one step to the next.

I do a lot of type-directed derivations of this style while I program in Haskell, with the type-checker checking each step for me. I’d love to have mechanized help in creating these derivations, not just checking them.

Zippers

How does the Holey class relate to zippers? As in a few recent blog posts, let’s use the fact that regular data types are isomorphic to fixed-points of functors.

Functor fixed-points are like function fixed points

fix f = f (fix f)

type Fix f = f (Fix f)

However, Haskell doesn’t support recursive type synonyms, so use a newtype:

newtype Fix f = Fix { unFix :: f (Fix f) }

A context for a functor fixed-point is either empty, if we’re at the very top of an “f-tree”, or it’s an f-context for f (Fix f), and a parent context:

data Context f = TopC | Context (Der f (Fix f)) (Context f)  -- first try

Hm. On the outside, Context f looks like a list, so let’s use a list instead:

type Context f = [Der f (Fix f)]

The location type we used above is

type Loc f a = (Der f a, a)

Similarly, define a type of zippers (also called “locations”) for functor fixed-points:

type Zipper f = (Context f, Fix f)

This Zipper type corresponds to a zipper, and has operations up and down. The down motion can yield multiple results.

up   :: Holey f ⇒ Zipper f →    Zipper f

down :: Holey f ⇒ Zipper f → f (Zipper f)

Since down yields an f-collection of locations, we do not need sibling navigation functions (left & right).

To move up in Zipper, strip off a derivative (one-hole functor context) and fill the hole with the current tree, leaving the other derivatives as the remaining fixed-point context. Like so:

up   :: Holey f ⇒ Zipper f →    Zipper f
up (d:ds', t) = (ds', Fix (fill (d,t)))

To see how the typing works out:

(d:ds', t) :: Zipper f
(d:ds', t) :: (Context f, Fix f)

d:ds' :: [Der f (Fix f)]

t :: Fix f

d   ::  Der f (Fix f)
ds' :: [Der f (Fix f)]

fill :: Loc f b → f b
fill :: (Der f b, b) → f b
fill :: (Der f (Fix f), Fix f) → f (Fix f)

fill (d,t) :: f (Fix f)

Fix (fill (d,t)) :: Fix f

(ds', Fix (fill (d,t))) :: (Context f, Fix f)
                        :: Zipper f

Note that the up motion fails when at the top of a zipper (empty context). If desired, we can also provide an unfailing version (really, a version with explictly typed failure):

up' :: Holey f => Zipper f -> Maybe (Zipper f)
up' ([]   , _) = Nothing
up' l          = Just (up l)

To move down in an f-tree t, form the extractions of t, each of which has a derivative and a sub-tree. The derivative becomes part of an extended fixed-point context, and the sub-tree becomes the new sub-tree of focus.

down :: Holey f ⇒ Zipper f → f (Zipper f)
down (ds', t) = (fmap.first) (:ds') (extract (unFix t)) (unFix t))

The typing (in case you’re curious):

(ds',t) :: Zipper f
        :: (Context f, Fix f)
        :: ([Der f (Fix f)], Fix f)

ds' :: [Der f (Fix f)]
t :: Fix f
unFix t :: f (Fix f)

extract (unFix t) :: f (Der f (Fix f), Fix f)

(fmap.first) (:ds') (extract (unFix t))
  :: ([Der f (Fix f)], Fix f)
  :: (Context f, Fix f)
  :: LocFix f

Zipping back to regular data types

I like the (functor) fixed-point perspective on regular data types, for its austere formal simplicity. It shows me the naked essence of regular data types, so I can more easily see and more deeply understand patterns like memoization, derivatives, and zippers.

For convenience and friendliness of use, I prefer working with regular types directly, rather than through the (nearly) isomorphic form of functor fixed-points. While the fixed-point perspective is formalism-friendly, the pattern functor perspective is more user-friendly, allowing us to work with our familiar regular data as they are.

As in Elegant memoization with higher-order types, let’s use the following class:

class Functor (PF t) ⇒ Regular t where
  type PF t :: * → *
  wrap   :: PF t t → t
  unwrap :: t → PF t t

The idea is that a type t is isomorphic to Fix (PF t), although really there may be more points of undefinedness in the fixed-point representation, so rather than an isomorphism, we have an embedding/projection pair.

The notions of context and location are similar to the ones above:

type Context t = [Der (PF t) t]

type Zipper t = (Context t, t)

So are the up and down motions, in which wrap and unwrap replace Fix and unFix:

up   :: (Regular t, Holey (PF t)) ⇒ Zipper t →       Zipper t
down :: (Regular t, Holey (PF t)) ⇒ Zipper t → PF t (Zipper t)

up (d:ds', t) = (ds', wrap (fill (d,t)))

down (ds', t) = (fmap.first) (:ds') (extract (unwrap t))

A “one-hole context” is a data structure with one piece missing. Conor McBride pointed out that the derivative of a regular type is its type of one-hole contexts. When a data structure is assembled out of common functor combinators, a corresponding type of one-hole contexts can be derived mechanically by rules that mirror the standard derivative rules learned in beginning differential calculus.

I’ve been playing with functor combinators lately. I was delighted to find that the data-structure derivatives can be expressed directly using the standard functor combinators and type families.

The code in this blog post is taken from the Haskell library functor-combo.

See also the Haskell Wikibooks page on zippers, especially the section called “Differentiation of data types”.

I mean this post not as new research, but rather as a tidy, concrete presentation of some of Conor’s delightful insight.

Functor combinators

Let’s use the same set of functor combinators as in Elegant memoization with higher-order types and Memoizing higher-order functions:

data Void a   -- no constructors

type Unit a        = Const () a

data Const x a     = Const x

newtype Id a       = Id a

data (f :+: g) a   = InL (f a) | InR (g a)

data (f :*: g) a   = f a :*: g a

newtype (g :. f) a = O (g (f a))

Derivatives

The derivative of a functor is another functor. Since the shape of the derivative is non-uniform (depends on the shape of the functor being differentiated) define a higher-order type family:

type family Der (f :: (* → *)) :: (* → *)

The usual derivative rules can then be translated without applying much imagination. That is, if we start with derivative rules in their functional form (e.g., as in the paper Beautiful differentiation, Section 2 and Figure 1).

For instance, the derivative of the constant function is the constant 0 function, and the derivative of the identity function is the constant 1 function. If der is the derivative functional mapping functions (of real numbers) to functions,

der (const x) ≡ 0
der id        ≡ 1

On the right-hand sides, I am exploiting the function instances of Num from the library applicative-numbers. To be more explicit, I could have written “const 0” and “const 1“.

Correspondingly,

type instance Der (Const x) = Void   -- 0

type instance Der Id        = Unit   -- 1

Note that the types Void a and Unit a have 0 and 1 element, respectively, if we ignore ⊥. Moreover, Void is a sort of additive identity, and Unit is a sort of multiplicative identity, again ignoring ⊥. For these reasons, Void and Unit might be more aptly named “Zero” and “One“.

The first rule says that the a value of type Const x a has no one-hole context (for type a), which is true, since there is an x but no a. The second rule says that there is exactly one possible context for Id a, since the one and only a value must be removed, and no information remains.

A (one-hole) context for a sum is a context for the left or the right possibility of the sum:

type instance Der (f :+: g) = Der f :+: Der g

Correspondingly, the derivative of a sum of functions is the sum of the functions’ derivatives::

der (f + g) ≡ der f + der g

Again I’m using the function Num instance from applicative-numbers.

For a pair, the one hole of a context can be made somewhere in the first component or somewhere in the second component. So the pair context consists of a holey first component and a full second component or a full first component and a holey second component.

type instance Der (f :*: g) = Der f :*: g  :+:  f :*: Der g

Similarly, for functions:

der (f * g) ≡ der f * g + f * der g

Finally, consider functor composition. If g and f are container types, then (g :. f) a is the type of g containers of f containers of a elements. The a-shaped hole must come from one of the contained f a structures.

type instance Der (g :. f) = (Der g :. f) :*: Der f

Here’s one way to think of this derivative functor: to make an a-shaped hole in a g (f a), first remove an f a structure, leaving an (f a)-shaped hole, and then put back all but an a value extracted from the removed f a struture. So the overall (one-hole) context can be assembled from two parts: a g context of f a structures, and an f context of a values.

The corresponding rule for function derivatives:

der (g ∘ f) ≡ (der g ∘ f) * der f

which again uses Num on functions. Written out more explicitly:

der (g ∘ f) a ≡ der g (f a)  * der f a

which may look more like the form you’re used to.

Summary of derivatives

To emphasize the correspondence between forms of differentiation, here are rules for function and functor derivatives:

der (const x) ≡ 0
Der (Const x) ≡ Void

der id ≡ 1
Der Id ≡ Unit

der (f  +  g) ≡ der f  +  der g
Der (f :+: g) ≡ Der f :+: Der g

der (f  *  g) ≡ der f  *  g  +  f  *  der g
Der (f :*: g) ≡ Der f :*: g :+: f :*: Der g

der (g  ∘ f) ≡ (der g  ∘ f)  *  der f
Der (g :. f) ≡ (Der g :. f) :*: Der f

Filling holes

Each derivative functor is a one-hole container. One useful operation on derivatives is filling that hole.

fillC :: Functor f ⇒ Der f a → a → f a

The specifics of how to fill in a hole will depend on the choice of functor f, so let’s make the fillC operation a method of a new type class. This new class is also a handy place to stash the associated type of derivatives, as an alternative to the top-level declarations above.

class Functor f ⇒ Holey f where
  type Der f :: * → *
  fillC :: Der f a → a → f a

I’ll add one more method to this class in an upcoming post.

For Const x, there are no cases to handle, since there are no holes.

instance Holey (Const x) where
  type Der (Const x) = Void
  fillC = error "fillC for Const x: no Der values"

I added a definition just to keep the compiler from complaining. This particular fillC can only be applied to a value of type Void a, and there are no such values other than ⊥.

Is there a more elegant way to define functions over data types with no constructors? One idea is to provide a single, polymorphic function over void types:

  voidF :: Void a → b
  voidF = error "voidF: no value of type Void"

And use whenever as needed, e.g.,

  fillC = voidF

Next is our identity functor:

instance Holey Id where
  type Der Id = Unit
  fillC (Const ()) a = Id a

More succinctly,

  fillC (Const ()) = Id

For sums,

instance (Holey f, Holey g) ⇒ Holey (f :+: g) where
  type Der (f :+: g) = Der f :+: Der g
  fillC (InL df) a = InL (fillC df a)
  fillC (InR df) a = InR (fillC df a)

or

  fillC (InL df) = InL ∘ fillC df
  fillC (InR df) = InR ∘ fillC df

Products also have two cases, since the derivative of a product is a sum:

instance (Holey f, Holey g) ⇒ Holey (f :*: g) where
  type Der (f :*: g) = Der f :*: g  :+:  f :*: Der g
  fillC (InL (dfa :*:  ga)) a = fillC dfa a :*: ga
  fillC (InR ( fa :*: dga)) a = fa :*: fillC dga a

Less pointfully,

  fillC (InL (dfa :*:  ga)) = (:*: ga) ∘ fillC dfa
  fillC (InR ( fa :*: dga)) = (fa :*:) ∘ fillC dga

Finally, functor composition:

instance (Holey f, Holey g) ⇒ Holey (g :. f) where
  type Der (g :. f) = (Der g :. f) :*: Der f
  fillC (O dgfa :*: dfa) a = O (fillC dgfa (fillC dfa a))

The less pointful form is more telling.

  fillC (O dgfa :*: dfa) = O ∘ fillC dgfa ∘ fillC dfa

In words: filling of the derivative of a composition is a composition of filling of the derivatives.

Thoughts on composition

Let’s return to the derivative rules for composition, i.e., the chain rule, on functions and on functors:

der (g  ∘ f) ≡ (der g  ∘ f)  *  der f

Der (g :. f) ≡ (Der g :. f) :*: Der f

Written in this way, the functor rule looks quite compelling. Something bothers me, however. For functions, multiplication is a special case, not the general case, and is only meaningful and correct when differentiating functions from scalars to scalars. In general, derivative values are linear maps, and the chain rule uses composition on linear maps rather than multiplication on scalars (that represent linear maps). I’ve written several posts on derivatives and a paper Beautiful differentiation, describing this perspective, which comes from calculus on manifolds.

Look again at the less pointful formulation of fillC for derivatives of compositions:

  fillC (O dgfa :*: dfa) = O ∘ fillC dgfa ∘ fillC dfa

The product in this case is just structural. The actual use in fillC is indeed a composition of linear maps. In this context, “linear” has a different meaning from before. It’s another way of saying “fills a one-hole context” (as the linear patterns of term rewriting and of ML & Haskell).

So maybe there’s a more general/abstract view of functor derivatives, just as there is a more general/abstract view of function derivatives. In that view, we might replace the functor chain rule’s product with a notion of composition.

The post Sequences, streams, and segments offered an answer to the the question of what’s missing in the following box:

I presented a simple type of function segments, whose representation contains a length (duration) and a function. This type implements most of the usual classes: Monoid, Functor, Zip, and Applicative, as well Comonad, but not Monad. It also implements a new type class, Segment, which generalizes the list functions length, take, and drop.

The function type is simple and useful in itself. I believe it can also serve as a semantic foundation for functional reactive programming (FRP), as I’ll explain in another post. However, the type has a serious performance problem that makes it impractical for some purposes, including as implementation of FRP.

Fortunately, we can solve the performance problem by adding a simple layer on top of function segments, to get what I’ll call “signals”. With this new layer, we have an efficient replacement for function segments that implements exactly the same interface with exactly the same semantics. Pleasantly, the class instances are defined fairly simply in terms of the corresponding instances on function segments.

You can download the code for this post.

Edits:

• 2008-12-06: dup [] = [] near the end (was [mempty]).
• 2008-12-09: Fixed take and drop default definitions (thanks to sclv) and added point-free variant.
• 2008-12-18: Fixed appl, thanks to sclv.
• 2011-08-18: Eliminated accidental emoticon in the definition of dup, thanks to anonymous.

The problem with function segments

The type of function segments is defined as follows:

data t :-># a = FS t (t -> a)

The domain of the function segment is from zero up to but not including the given length.

An efficiency problem becomes apparent when we look at the Monoid instance:

instance (Ord t, Num t) => Monoid (t :-># a) where
    mempty = FS 0 (error "sampling empty 't :-># a'")
    FS c f `mappend` FS d g =
      FS (c + d) ( t -> if t <= c then f t else g (t - c))

Concatenation (mappend) creates a new segment that chooses, for every domain value t, whether to use one function or another. If the second, then t must be shifted backward, since the function is being shifted forward.

This implementation would be fine if we use mappend just on simple segments. Once we get started, however, we’ll want to concatenate lots & lots of segments. In FRP, time-varying values go through many phases (segments) as time progresses. Each quantity is described by a single “behavior” (sometimes called a “signal”) with many, and often infinitely many, phases. Imagine an infinite tree of concatenations, which is typical for FRP behaviors. At every moment, one phase is active. Every sampling must recursively discover the active phase and the accumulated domain translation (from successive subtractions) to apply when sampling that phase. Quite commonly, concatenation trees get progressively deeper on the right (larger t values). In that case, sampling will get slower and slower with time.

I like to refer to these progressive slow-downs as “time leaks”. There is also a serious space leak, since all of the durations and functions that go into a composed segment will be retained.

Sequences of segments

The problem above can be solved with a simple representation change. Instead of combining functions into functions, just keep a list of simple function segments.

-- | Signal indexed by t with values of type a.
newtype t :-> a = S { unS :: [t :-># a] }

I’ll restrict in function segments to be non-empty, to keep the rest of the implementation simple and efficient.

This new representation allows for efficient monotonic sampling of signals. As old segments are passed up, they can be dropped.

What does it mean?

There’s one central question for me in defining any data type: What does it mean?

The meaning I’ll take for signals is function segments. This interpretation is made precise in a function that maps from the type to the model (meaning). In this case, simply concatenate all of the function segments:

meaning :: (Ord t, Num t) => (t :-> a) -> (t :-># a)
meaning = mconcat . unS

Specifying the meaning of a type gives users a working model, and it defines correctness of implementation. It also tells me what class instances to implement and tells users what instances to expect. If a type’s meaning implements a class then I want the type to as well. Moreover, the type’s intances have to agree with the model’s instances. I’ve described this latter principle in Simplifying semantics with type class morphisms and some other posts.

Higher-order wrappers

To keep the code below short and clear, I’ll use some functions for adding and removing the newtype wrappers. These higher-order function apply functions inside of (:->) representations:

inS  :: ([s :-># a] -> [t :-># b])
     -> ((s :->  a) -> (t :->  b))

inS2 :: ([s :-># a] -> [t :-># b] -> [u :-># c])
     -> ((s :->  a) -> (t :->  b) -> (u :->  c))

Using the trick described in Prettier functions for wrapping and wrapping, the definitions are simpler than the types:

inS  = result   S . argument unS
inS2 = result inS . argument unS

Functor

The Functor instance applies a given function inside the function segments inside the lists:

instance Functor ((:->) t) where
    fmap h (S ss) = S (fmap (fmap h) ss)

Or, in the style of Semantic editor combinators,

instance Functor ((:->) t) where
    fmap = inS . fmap . fmap

Why this definition? Because it is correct with respect to the semantic model, i.e., the meaning of fmap is fmap of the meaning, i.e.,

 meaning . fmap h == fmap h . meaning

which is to say that the following diagram commutes:

Proof:

meaning . fmap h 

  == {- fmap definition -}

meaning . S . fmap (fmap h) . unS

  == {- meaning definition -}

mconcat . unS . S . fmap (fmap h) . unS

  == {- unS and S are inverses  -}

mconcat . fmap (fmap h) . unS

  == {- fmap h distributes over mappend -}

fmap h . mconcat . unS

  == {- meaning definition -}

fmap . meaning

Applicative and Zip

Again, the meaning functions tells us what the Applicative instance has to mean. We only get to choose how to implement that meaning correctly. The Applicative morphism properties:

meaning (pure a)    == pure a
meaning (bf <*> bx) == meaning bf <*> meaning bx

Our Applicative instance has a definition similar in simplicity and style to the Functor instance, assuming a worker function appl for (<*>):

instance (Ord t, Num t, Bounded t) => Applicative ((:->) t) where
    pure  = S . pure . pure
    (<*>) = inS2 appl

appl :: (Ord t, Num t, Bounded t) =>
        [t :-># (a -> b)] -> [t :-># a] -> [t :-># b]

This worker function is somewhat intricate. At least my implementation of it is, and perhaps there’s a simpler one.

Again, the meaning functions tells us what the Applicative instance has to mean. We only get to choose how to implement that meaning correctly.

First, if either segment list runs out, the combination runs out (because the same is true for the meaning of signals).

[] `appl` _  = []

_  `appl` [] = []

If neither segment list is empty, open up the first segment. Split the longer segment into a prefix that matches the shorter segment, and combine the two segments with (<*>) (on function segments). Toss the left-over piece back in its list, and continue.

(fs:fss') `appl` (xs:xss')
   | fd == xd  = (fs  <*> xs ) : (fss' `appl`       xss' )
   | fd <  xd  = (fs  <*> xs') : (fss' `appl` (xs'':xss'))
   | otherwise = (fs' <*> xs ) : ((fs'':fss') `appl` xss')
 where
   fd         = length fs
   xd         = length xs
   (fs',fs'') = splitAt xd fs
   (xs',xs'') = splitAt fd xs

A Zip instance is easy as always with applicative functors:

instance (Ord t, Num t, Bounded t) => Zip ((:->) t) where zip = liftA2 (,)

Monoid

The Monoid instance:

instance Monoid (t :-> a) where
  mempty = S []
  S xss `mappend` S yss = S (xss ++ yss)

Correctness follows from properties of mconcat (as used in the meaning function).

We’re really just using the Monoid instance for the underlying representation, i.e.,

instance Monoid (t :-> a) where
    mempty  = S mempty
    mappend = inS2 mappend

Segment

The Segment class has length, take and drop. It’s handy to include also null and splitAt, both modeled after their counterparts on lists.

The new & improved Segment class:

class Segment seg dur | seg -> dur where
    null    :: seg -> Bool
    length  :: seg -> dur
    take    :: dur -> seg -> seg
    drop    :: dur -> seg -> seg
    splitAt :: dur -> seg -> (seg,seg)
    -- Defaults:
    splitAt d s = (take d s, drop d s)
    take    d s = fst (splitAt d s)
    drop    d s = snd (splitAt d s)

If we wanted to require dur to be numeric, we could add a default for null as well. This default could be quite expensive in some cases. (In the style of Semantic editor combinators, take = (result.result) fst splitAt, and similarly for drop.)

The null and length definitions are simple, following from to properties of mconcat.

instance (Ord t, Num t) => Segment (t :-> a) t where
  null   (S xss) = null xss
  length (S xss) = sum (length <$> xss)
  ...

The null case says that the signal is empty exactly when there are no function segments. This simple definition relies on our restriction to non-empty function segments. If we drop that restriction, we’d have to check that every segment is empty:

  -- Alternative definition
  null (S xss) = all null xss

The length is just the sum of the lengths.

The tricky piece is splitAt (used to define both take and drop), which must assemble segments to satisfy the requested prefix length. The last segment used might have to get split into two, with one part going into the prefix and one to the suffix.

  splitAt _ (S [])  = (mempty,mempty)
  splitAt d b | d <= 0 = (mempty, b)
  splitAt d (S (xs:xss')) =
    case (d `compare` xd) of
      EQ -> (S [xs], S xss')
      LT -> let (xs',xs'') = splitAt d xs in
              (S [xs'], S (xs'':xss'))
      GT -> let (S uss, suffix) = splitAt (d-xd) (S xss') in
              (S (xs:uss), suffix)
   where
     xd = length xs

Copointed and Comonad

To extract an element from a signal, extract an element from its first function segment. Awkwardly, extraction will fail (produce ⊥/error) when the signal is empty.

instance Num t => Copointed ((:->) t) where
    extract (S [])     = error "extract: empty S"
    extract (S (xs:_)) = extract xs

I’ve exploited our restriction to non-empty function segments. Otherwise, extract would have to skip past the empty ones:

-- Alternative definition
instance Num t => Copointed ((:->) t) where
    extract (S []) = error "extract: empty S"
    extract (S (xs:xss'))
      | null xs     = extract (S xss')
      | otherwise   = extract xs

The error/⊥ in this definition is dicey, as is the one for function segments. If we allow the same abuse in order to define a list Copointed, we can get an alternative to the first definition that is prettier but gives a less helpful error message:

instance Num t => Copointed ((:->) t) where
    extract = extract . extract . unS

See the closing remarks for more about this diciness.

Finally, we have Comonad, with its duplicate method.

duplicate :: (t :-> a) -> (t :-> (t :-> a))

I get confused with wrapping and unwrapping, so let’s separate the definition into a packaging part and a content part.

instance (Ord t, Num t) => Comonad ((:->) t) where
    duplicate = fmap S . inS dup

with content part:

dup :: (Ord t, Num t) => [t :-># a] -> [t :-># [t :-># a]]

The helper function, dup, takes each function segment and prepends each of its tails onto the remaining list of segments.

If the segment list is empty, then it has only one tail, also empty.

dup []        = []

dup (xs:xss') = ((: xss') <$> duplicate xs) : dup xss'

Closing remarks

• The definitions above use the function segment type only through its type class interfaces, and so can all of them can be generalized. Several definitions rely on the Segment instance, but otherwise, each method for the composite type relies on the corresponding method for the underlying segment type. (For instance, fmap uses fmap, (<*>) uses (<*>), Segment uses Segment, etc.) This generality lets us replace function segments with more efficient representations, e.g., doing constant propagation, as in Reactive. We can also generalize from lists in the definitions above.

• Even without concatenation, function segments can become expensive when drop is repeatedly applied, because function shifts accumulate (e.g., f . (+ 0.01) . (+ 0.01) ....). A more drop-friendly representation for function segments would be a function and an offset. Successive drops would add offsets, and extract would always apply the function to its offset. This representation is similar to the FunArg comonad, mentioned in The Essence of Dataflow Programming (Section 5.2).

• The list-of-segments representation enables efficient monotonic sampling, simply by dropping after each sample. A variation is to use a list zipper instead of a list. Then non-monotonic sampling will be efficient as long as successive samplings are for nearby domain values. Switching to multi-directional representation would lose the space efficiency of unidirectional representations. The latter work well with lazy evaluation, often running in constant space, because old values are recycled while new values are getting evaluated.

• Still another variation is to use a binary tree instead of a list, to avoid the list append in the Monoid instance for (t :-> a). A tree zipper would allow non-monotonic sampling. Sufficient care in data structure design could perhaps yield efficient random access.

• There’s a tension between the Copointed and Monoid interfaces. Copointed has extract, and Monoid has mempty, so what is the value of extract mempty? Given that Copointed is parameterized over arbitrary types, the only possible answer seems to be ⊥ (as in the use of error above). I don’t know if the comonad laws can all hold for possibly-empty function segments or for possibly-empty signals. I’m grateful to Edward Kmett for helping me understand this conflict. He suggested using two coordinated types, one possibly-empty (a monoid) and the other non-empty (a comonad). I’m curious to see whether that idea works out.