24th February 2009, 12:05 am

I have another paper draft for submission to ICFP 2009.
This one is called *Beautiful differentiation*,
The paper is a culmination of the several posts I’ve written on derivatives and automatic differentiation (AD).
I’m happy with how the derivation keeps getting simpler.
Now I’ve boiled extremely general higher-order AD down to a `Functor`

and `Applicative`

morphism.

I’d love to get some readings and feedback.
I’m a bit over the page the limit, so I’ll have to do some trimming before submitting.

The abstract:

Automatic differentiation (AD) is a precise, efficient, and convenient
method for computing derivatives of functions. Its implementation can be
quite simple even when extended to compute all of the higher-order
derivatives as well. The higher-dimensional case has also been tackled,
though with extra complexity. This paper develops an implementation of
higher-dimensional, higher-order differentiation in the extremely
general and elegant setting of *calculus on manifolds* and derives that
implementation from a simple and precise specification.

In order to motivate and discover the implementation, the paper poses
the question “What does AD mean, independently of implementation?” An
answer arises in the form of *naturality* of sampling a function and its
derivative. Automatic differentiation flows out of this naturality
condition, together with the chain rule. Graduating from first-order to
higher-order AD corresponds to sampling all derivatives instead of just
one. Next, the notion of a derivative is generalized via the notions of
vector space and linear maps. The specification of AD adapts to this
elegant and very general setting, which even *simplifies* the
development.

You can get the paper and see current errata here.

The submission deadline is March 2, so comments before then are most helpful to me.

Enjoy, and thanks!

I have another paper draft for submission to ICFP 2009. This one is called Beautiful differentiation, The paper is a culmination of the several posts I’ve written on derivatives and...

18th February 2009, 06:34 pm

I’ve just finished a draft of a paper called *Denotational design with type class morphisms*, for submission to ICFP 2009.
The paper is on a theme I’ve explored in several posts, which is semantics-based design, guided by type class morphisms.

I’d love to get some readings and feedback.
Pointers to related work would be particularly appreciated, as well as what’s unclear and what could be cut.
It’s an entire page over the limit, so I’ll have to do some trimming before submitting.

The abstract:

Type classes provide a mechanism for varied implementations of standard
interfaces. Many of these interfaces are founded in mathematical
tradition and so have regularity not only of *types* but also of
*properties* (laws) that must hold. Types and properties give strong
guidance to the library implementor, while leaving freedom as well. Some
of the remaining freedom is in *how* the implementation works, and some
is in *what* it accomplishes.

To give additional guidance to the *what*, without impinging on the
*how*, this paper proposes a principle of *type class morphisms* (TCMs),
which further refines the compositional style of denotational
semantics. The TCM idea is simply that *the instance’s meaning is the
meaning’s instance*. This principle determines the meaning of each type
class instance, and hence defines correctness of implementation. In some
cases, it also provides a systematic guide to implementation, and in
some cases, valuable design feedback.

The paper is illustrated with several examples of type, meanings, and
morphisms.

You can get the paper and see current errata here.

The submission deadline is March 2, so comments before then are most helpful to me.

Enjoy, and thanks!

I’ve just finished a draft of a paper called Denotational design with type class morphisms, for submission to ICFP 2009. The paper is on a theme I’ve explored in several...

8th February 2009, 02:14 pm

In *What is automatic differentiation, and why does it work?*, I gave a semantic model that explains what automatic differentiation (AD) accomplishes.
Correct implementations then flowed from that model, by applying the principle of type class morphisms.
(An instance’s interpretation is the interpretation’s instance).

I’ve had a nagging discomfort about the role of the chain rule in AD, with an intuition that the chain rule can carry a more central role the the specification and implementation.
This post gives a variation on the previous AD post that carries the chain rule further into the reasining and implementation, leading to simpler correctness proofs and a nearly unaltered implementation.

Finally, as a bonus, I’ll show how GHC rewrite rules enable an even simpler and more modular implementation.

I’ve included some optional content, including exercises.
You can see my answers to the exercises by examining the HTML.

Continue reading ‘From the chain rule to automatic differentiation’ »

In What is automatic differentiation, and why does it work?, I gave a semantic model that explains what automatic differentiation (AD) accomplishes. Correct implementations then flowed from that model, by...

3rd February 2009, 04:47 pm

Thanks to my time at Microsoft Research (1994-2002), I was able to live for a while (modestly) on asset growth.
Last year I started working in earnest on two libraries, Reactive (functional reactive programming) and FieldTrip (functional 3D), plus some supporting libraries.
I placed these libraries all under the most liberal license I know, which is BSD3.
I’m more enthusiastic than ever about my functional programming work and am enjoying active involvement with the Haskell community.

With the recent economic meltdown, my old means of sustainable income ended, and now I’m looking for a replacement.
I’m not yet in crisis, so I have time to make some thoughtful decisions and even take some risk.
Rather than abandoning Reactive, FieldTrip, and related projects (some new), I’m looking for ways to continue these projects while building potential for future income related to them.
At the same time, it’s very important to me to keep these projects open so as to advance purely functional techniques & tools, as well as to have enjoyable creative connections, and to get feedback & help.

For these reasons, I’m now considering licensing future releases of some my libraries for *non-commercial* use, with commercial uses to be arranged as separate licenses.
I know almost nothing about licensing issues, because I haven’t been interested, and I’ve always wanted maximum freedom for all users.

So, I’m looking for help in choosing a software license that enables & encourages a creative community, while preserving opportunity to ask for some portion of return in future for-profit uses.
If people have alternative perspectives how to achieve my goals *without* changing license terms, I’m interested in hearing them as well.
I am not trying to “make a killing”, just a living, so that I can keep doing what I love and share it with others.

Thanks to my time at Microsoft Research (1994-2002), I was able to live for a while (modestly) on asset growth. Last year I started working in earnest on two libraries,...